Disability Employment Initiative Project Implementation and Impact Evaluation
Round 5 through Round 6 Grantees
Project Director
Douglas Klayman, Ph.D.
UAUTHORS
Social Dynamics, LLC
Gary Shaheen, Ph.D.
Douglas Klayman, Ph.D.
Aaron Searson, Ph.D.
Amy Rowland, MFA
Melissa Williamson, A.A.
Abt Associates
Sung-Woo Cho, Ph.D., Statistician
March 19, 2020
Submitted to:
U.S. Department of Labor
Office of Disability Employment Policy
200 Constitution Avenue, NW
Washington, DC 20210
Submitted by:
Social Dynamics, LLC
This report was prepared for the U.S. Department of Labor
(USDOL), Office of Disability Employment Policy, by Social
Dynamics, LLC, and Abt Associates under Task Order
Contract #DOL- OPS-15-A-0015. The views expressed are
those of the authors and should not be attributed to USDOL,
nor does mention of trade names, commercial products, or
organizations imply endorsement of same by the U.S.
Government.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
1
Table of Contents
I. EXECUTIVE SUMMARY .................................................................................................................................. 5
II. INTRODUCTION ............................................................................................................................................ 9
A. DISABILITY AND LABOR FORCE ENGAGEMENT ........................................................................................................ 9
III. DESCRIPTION OF THE PROGRAM ................................................................................................................ 10
IV. EVALUATION DESIGN .................................................................................................................................. 13
V. PROGRAM IMPLEMENTATION .................................................................................................................... 14
A. IMPLEMENTATION OF DEI SERVICE DELIVERY STRATEGIES ..................................................................................... 14
B. FINDINGS FROM THE SOCIAL DYNAMICS SYSTEMS CHANGE CODING RUBRIC ............................................................. 28
C. DEVELOPING DOMAINS AND INDICATORS FOR SYSTEMS CHANGE ............................................................................ 29
VI. GRANT IMPLEMENTATION ROUND 5 .......................................................................................................... 36
A. STATE: CALIFORNIA; FOCUS AREA: ADULTS WITH DISABILITIES ............................................................................... 36
i. Stated Goals and Objectives ..................................................................................................................... 36
ii. DEI Service Delivery Strategies ................................................................................................................. 37
iii. Implementation Summary ........................................................................................................................ 38
iv. Partnerships and Collaborations............................................................................................................... 39
v. Career Pathways ....................................................................................................................................... 41
vi. Outreach and Dissemination .................................................................................................................... 41
vii. Promising Practices: “Pathways for Success” ........................................................................................... 41
viii. Challenges and Sustainability .............................................................................................................. 43
B. STATE: ILLINOIS; FOCUS AREA: ADULTS WITH DISABILITIES .................................................................................... 44
i. Stated Goals and Objectives ..................................................................................................................... 44
ii. DEI Service Delivery Strategies ................................................................................................................. 44
iii. Implementation Summary ........................................................................................................................ 45
iv. Partnerships and Collaborations............................................................................................................... 46
v. Employer Partnership ............................................................................................................................... 47
vi. Career Pathways ....................................................................................................................................... 48
vii. Outreach and Dissemination .................................................................................................................... 49
viii. Promising Practices: “Pathways for Success” ...................................................................................... 50
ix. Challenges and Sustainability ................................................................................................................... 50
C. STATE: KANSAS; FOCUS AREA: ADULTS WITH DISABILITIES ..................................................................................... 51
i. Stated Goals and Objectives ..................................................................................................................... 51
ii. DEI Service Delivery Strategies ................................................................................................................. 52
iii. Implementation Summary ........................................................................................................................ 53
iv. Partnerships and Collaborations............................................................................................................... 54
v. Career Pathways ....................................................................................................................................... 57
vi. Outreach and Dissemination .................................................................................................................... 57
vii. Promising Practices: “Pathways for Success” ........................................................................................... 58
viii. Challenges and Sustainability .............................................................................................................. 58
D. STATE: MASSACHUSETTS; FOCUS AREA: ADULTS WITH DISABILITIES ........................................................................ 60
i. Stated Goals and Objectives ..................................................................................................................... 60
ii. DEI Service Delivery Strategies ................................................................................................................. 61
iii. Implementation Summary ........................................................................................................................ 62
iv. Partnerships and Collaborations............................................................................................................... 63
v. Career Pathways ....................................................................................................................................... 69
vi. Outreach and Dissemination .................................................................................................................... 73
vii. Promising Practices: “Pathways for Success” ........................................................................................... 73
viii. Challenges and Sustainability .............................................................................................................. 74
E. STATE: MINNESOTA; FOCUS AREA: ADULTS WITH DISABILITIES .............................................................................. 77
i. Stated Goals and Objectives ..................................................................................................................... 77
ii. DEI Service Delivery Strategies ................................................................................................................. 77
iii. Implementation Summary ........................................................................................................................ 81
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
2
iv. Partnerships and Collaborations............................................................................................................... 82
v. Career Pathways ....................................................................................................................................... 86
vi. Outreach and Dissemination .................................................................................................................... 88
vii. Promising Practices: “Peer Mentoring” .................................................................................................... 89
viii. Challenges and Sustainability .............................................................................................................. 90
F. STATE: SOUTH DAKOTA; FOCUS AREA: ADULTS WITH DISABILITIES .......................................................................... 92
i. Stated Goals and Objectives ..................................................................................................................... 92
ii. DEI Service Delivery Strategies ................................................................................................................. 93
iii. Implementation Summary ........................................................................................................................ 94
iv. Partnerships and Collaborations............................................................................................................... 96
v. Career Pathways ....................................................................................................................................... 98
vi. Outreach and Dissemination .................................................................................................................... 99
vii. Promising Practices: “Employment Tours ............................................................................................. 100
viii. Challenges and Sustainability ............................................................................................................ 101
VII. GRANT IMPLEMENTATION ROUND 6 ................................................................................................... 102
A. STATE: ALASKA; FOCUS AREA: YOUTH WITH DISABILITIES .................................................................................... 102
i. Stated Goals and Objectives ................................................................................................................... 102
ii. DEI Service Delivery Strategies ............................................................................................................... 103
iii. Implementation Summary ...................................................................................................................... 105
iv. Partnerships and Collaborations............................................................................................................. 105
v. Career Pathways ..................................................................................................................................... 107
vi. Outreach and Dissemination .................................................................................................................. 108
vii. Promising Practices 1: “Training DRC 1s” ............................................................................................... 108
viii. Promising Practices 2: “Coordinated DEI and DVR Approaches to Serving Youth with Disabilities with
Criminal Justice Backgrounds ......................................................................................................................... 109
ix. Challenges and Sustainability ................................................................................................................. 110
B. STATE: GEORGIA; FOCUS AREA: YOUTH WITH DISABILITIES .................................................................................. 112
i. Stated Goals and Objectives ................................................................................................................... 112
ii. DEI Service Delivery Strategies ............................................................................................................... 113
iii. Implementation Summary ...................................................................................................................... 114
iv. Partnerships and Collaborations............................................................................................................. 115
v. Career Pathways ..................................................................................................................................... 117
vi. Outreach and Dissemination .................................................................................................................. 117
vii. Promising Practices “Pathways for Success” .......................................................................................... 118
viii. Challenges and Sustainability ............................................................................................................ 118
C. STATE: HAWAII; FOCUS AREA: INDIVIDUALS WITH SIGNIFICANT DISABILITIES ........................................................... 120
i. Stated Goals and Objectives ................................................................................................................... 120
ii. DEI Service Delivery Strategies ............................................................................................................... 120
iii. Implementation Summary ...................................................................................................................... 121
iv. Partnerships and Collaborations............................................................................................................. 124
v. Career Pathways ..................................................................................................................................... 126
vi. Outreach and Dissemination .................................................................................................................. 126
vii. Promising Practices................................................................................................................................. 127
viii. Challenges and Sustainability ............................................................................................................ 127
VIII. QUASI-EXPERIMENTAL METHODOLOGY ............................................... ERROR! BOOKMARK NOT DEFINED.
A. MATCHED COMPARISON GROUP ANALYSIS WHEN COMPARISON GROUPS EXIST ..................................................... 167
B. MATCHING USING TREATMENT AND COMPARISON GROUP INDIVIDUALS CHARACTERISTICS ...................................... 168
C. SHORT INTERRUPTED TIME SERIES (SITS) ANALYSIS WHEN NO COMPARISON GROUPS EXIST .................................... 169
D. MEASURING THE IMPACT OF THE CAREER PATHWAYS COMPONENT ...................................................................... 172
E. TREATMENT AND COMPARISON GROUP INDIVIDUALS IN THE CAREER PATHWAYS ANALYSIS ....................................... 173
IX. IMPACT ANALYSIS..................................................................................................................................... 173
A. DESCRIPTIVE STATISTICS ................................................................................................................................ 173
B. EMPLOYMENT/RETENTION/EARNINGS OUTCOMES ............................................................................................ 175
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
3
C. SURVEY TO COLLECT INFORMATION ON ACTIVITIES OF DAILY LIVING AND DISABILITY TYPE IN COMPARISON GROUP SITES 175
D. IMPACT ESTIMATES FOR MATCHED COMPARISON GROUP ANALYSIS ...................................................................... 177
E. OVERALL IMPACT OF DEI, WITH MATCHING ..................................................................................................... 179
F. IMPACT ESTIMATES FOR SHORT INTERRUPTED TIME SERIES (SITS) ANALYSIS .......................................................... 182
G. IMPACT OF CAREER PATHWAYS ...................................................................................................................... 187
H. IMPACT OF CAREER PATHWAYS, WITH MATCHING ............................................................................................. 190
X. FINDINGS AND DISCUSSION ...................................................................................................................... 193
A. POTENTIAL AREAS FOR FUTURE RESEARCH ....................................................................................................... 194
B. DEI EVALUATION LOGIC MODEL .................................................................................................................... 195
C. SUSTAINABILITY FINDINGS ............................................................................................................................. 213
APPENDIX 1: AMERICAN JOB CENTER SURVEY .......................................................................................................... 218
APPENDIX 2: DEI ROUND 5 AND ROUND 6 WDAS ..................................................................................................... 219
APPENDIX 3: ROUND 5 TREATMENT AND COMPARISON SITES FOR QED ANALYSIS................................................. 221
APPENDIX 4: ROUND 6 TREATMENT AND COMPARISON SITES FOR QED ANALYSIS................................................. 222
APPENDIX 4: ROUND 6 TREATMENT AND COMPARISON SITES FOR QED ANALYSIS................................................. 222
EXHIBIT 1: COMPLETE LIST OF SERVICE DELIVERY STRATEGIES ................................................................................... 11
EXHIBIT 2: UTILIZATION OF SERVICE DELIVERY STRATEGIES ....................................................................................... 12
EXHIBIT 3: ODEP-SUGGESTED DEI ROUND 5 AND ROUND 6 SERVICE DELIVERY STRATEGIES .................................... 21
EXHIBIT 4: PRE- AND POST-INTERVENTION TREND LINES TO MEASURE THE IMPACT OF AN INTERVENTION.….……171
EXHIBIT 5: DEI COMPARISON GROUP QED RAW SAMPLE CHARACTERISTICS-UNMATCHED………………………………….174
EXHIBIT 6: DEI COMPARISON GROUP QED RAW SAMPLE OUTCOMES-UMATCHED………………………………………………175
EXHIBIT 7: SURVEY CHARACTERISTICS OF INDIVIDUALS WITHIN COMPARISON SITES………………………………………..176
EXHIBIT 8: DEI COMPARISON GROUP QED IMPACT RESULTS AND CONTROL VARIABLE COEFFICIENTS
UMATCHED……………………………………………………………………………………………………………………………………………………….…178
EXHIBIT 9: DEI COMPARISON GROUP QED RAW SAMPLE CHARACTERISTICS Matched ……………………………………180
EXHIBIT 10: DEI COMPARISON GROUP QED IMPACT RESULTS AND CONTROL VARIABLE COEFFICIENTS
-MATCHED……………………………………………………………………………………………………………………………………………………………181
EXHIBIT 11: SITS SAMPLE CHARACTERISTICS (INDIVIDUALS IN TREATMENT WDAs PRE- AND POST DEI) …….….…184
EXHIBIT 12: DEI SITS IMPACT RESULTS AND CONTROL VARIABLE COEFFICIENTS………………………………………………....186
EXHIBIT 13: CAREER PATHWAYS COMPARISON GROUP QED RAW SAMPLLE CHARACTERISTICS UNMATCHED…188
EXHIBIT 14: CAREER PATHWAYS COMPARISON GROUP QED IMPACT RESULTS AND CONTROL VARIABLE
COEFFICIENTS-UNMATCHED…………………………………………………………………………………………………………………………………189
EXHIBIT 15: CAREER PATHWAYS COMPARISON GROUP QED RAW SAMPLE CHARACTERSTICS
— MATCHED…………………………………………………………………………………………………………………………………………………………191
EXHIBIT 16: CAREER PATHWAYS COMPARISON GROUP QED IMPACT RESULTS AND CONTROL VARIABLE
COEFFICIENTS — MATCHED……………………………………………………………………………………………………………………………….192
EXHIBIT 17: QUALITIVE RESEARCH QUESTIONS…………………………………………………………………………………………………….194
EXHIBIT 18: DEI LOGIC MODEL………………………………………………………………………………………………………………………………197
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
4
ACRONYMS
(ABE) Adult Basic Education
(ADA) Americans with Disabilities Act
(AJC) American Job Center
(ARC) Active Resource Coordination
(AT) Assistive Technology
(BLN) Business Leadership Network
(CE) Customized Employment
(CNA) Certified Nursing Assistant
(CP) Career Pathways
(CWIC) Community Work Incentives Coordinator
(DCS) MassHire Department of Career Services
(DEA) Disability Employment Accelerator
(DEI) Disability Employment Initiative
(DLR) South Dakota Department of Labor and
Regulation
(DOR) California Department of Rehabilitation
(DRC) Disability Resource Coordinator
(DVR) Department of Vocational Rehabilitation
(EN) Employment Network
(ETA) Employment and Training Administration
(GED) General Education Diploma
(GWDB) Governor’s Workforce Development
Board
(ICI) Institute for Community Inclusion
(ILP) Individual Learning Plan
(IRT) Integrated Resource Team
(IT) Information Technology
(JSWDs) Jobseekers with Disabilities
(MCB) Massachusetts Commission for the Blind
(MCDHH) Massachusetts Commission for the
Deaf and Hard of Hearing
(MOU) Memorandum of Understanding
(MRC) Massachusetts Rehabilitation Commission
(NDEAM) National Disability Employment
Awareness Month
(NDI) National Disability Institute
(ODEP) Office of Disability Employment Policy
(OJT) On-the-Job Training
(PASS) Plan for Achieving Self-Support
(PIRL) Participant Individual Record Layout
(QED) Quasi-Experimental Design
(SCCS) Systems Change Coding Scheme
(SDS) Service Delivery Strategy
(SITS) Short Interrupted Time Series
(SNAP) Supplemental Nutrition Assistance
Program
(SSA) United States Social Security
Administration
(SSDI) Social Security Disability Insurance
(SSI) Supplemental Security Income
(TANF) Temporary Assistance for Needy
Families
(TORQ) Transferable Occupation Relationship
Quotient
(T/TA) Training and Technical Assistance
(TTW) Ticket to Work
(USDOL) United States Department of Labor
(VR) Vocational Rehabilitation
(WBL) Work-Based Learning
(WDA) Workforce Development Area
(WDD) Hawaii Workforce Development Division
(WIA) Workforce Investment Act
(WIOA) Workforce Innovation and Opportunity
Act
(WIPA) Work Incentives Planning and Assistance
(WWL) Work Without Limits
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
5
I. Executive Summary
1. The Disability Employment Initiative (DEI) Round 5 and Round 6 evaluation was
conducted by Social Dynamics, LLC, and Abt Associates. It was designed to report to the
United States Department of Labor’s (USDOL) Office of Disability Employment Policy and
Employment and Training Administration on the implementation, outcomes, and impact of
the DEI. Research questions focused on employment outcomes and the impact of the project
on jobseekers with disabilities (JSWDs). Other research questions centered on service
delivery strategies (SDSs) and grant requirements that were embedded within the workforce
development systems of participating Workforce Development Areas (WDAs).
1
2. Grantees used a wide array of SDSs across treatment sites and exhibited considerable
variability in the way the DEI and grant-funded leadership positions were
implemented. A primary focus on the goal of employment for JSWDs was persistent
throughout. Unlike some earlier rounds, project implementation and engagement of JSWDs
was exceedingly precise, with few challenges, particularly in states that had prior DEI
experience. Staff turnover, retention, and contamination were not factors that, for most
grantees, affected the implementation or operation of the DEI. Those that experienced these
challenges were able to overcome them relatively quickly. At the same time, the level of
engagement of JSWDs appeared to be robust. With respect to certain prior rounds, we
characterized the DEI as a living laboratory of practical insight about the way JSWDs can
be served through the AJCs and their partners.”
2
DEI Round 5 and Round 6 functioned with
less variability than in prior rounds with clearly defined roles and a distribution of tasks. In
most Round 5 and Round 6 WDAs, DEI State Leads and Disability Resource Coordinators
(DRCs) knew who to report to, who made key decisions, and who provided guidelines
leading to project effectiveness.
3. The DEI was designed to improve the capacity of AJCs to increase employment
opportunities for people with disabilities. AJCs are primarily financed by the federal
government through funding streams that provide support services to veterans, youth,
dislocated workers, school dropouts, individuals in poverty, and individuals with disabilities.
DEI State Leads and DRCs worked in 12 (Round 5 and Round 6) states to build capacity for
the implementation of SDSs and training for staff and JSWDs and to foster opportunities for
partnerships and collaborations with WIOA providers and community-based agencies to
resolve barriers that inhibited access to needed services. DEI Round 5 and Round 6 grantees
administered SDSs with proficiency, augmented by a plethora of services and supports,
including enrollment in WIOA and Ticket to Work (TTW).
4. The DEI included two grant-funded leadership positions. Each grantee included a DEI
State Lead and one or more DRCs for each treatment WDA. These positions provided
executive leadership and expertise in workforce development, program implementation,
1
Throughout the text of this report, we refer to both WDAs and American Job Centers (AJCs). A single WDA may
include one or more AJCs that provide core, intensive, and/or Workforce Innovation and Opportunity Act (WIOA)
services.
2
Cori DiBiase, comment, March 9, 2020.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
6
SDSs, partnerships and collaborations, benefits counseling, TTW and Employment Network
(EN) program management, organizational development, and project sustainability.
5. The DEI State Lead provided executive leadership. DEI State Leads were responsible for
monitoring finances, developing state and local partnerships, and providing workforce
development and program implementation support. They also monitored ENs, conferred with
WDAs, managed the grant, and corresponded with the Federal Project Officers. DEI State
Leads also provided support for the implementation of DEI projects and DRCs.
6. DRCs were tasked with the implementation of SDSs in addition to partnership-
building, benefits counseling, TTW and EN program management, organizational
development, and project sustainability. DRCs who completed Community Work
Incentives Coordinator (CWIC) training provided benefits counseling to JSWDs. Other DRC
tasks supporting JSWDs included employment preparation, job placement support, soft skills
training, job coaching, leadership for Integrated Resource Teams and support services for
JSWDs as they enrolled in community colleges to improve their basic skills in math, reading,
or English prior to accepting an offer of employment. DRCs who had prior case management
experiences were often recognized for their ability to understand the needs of JSWDs and
respond effectively to those needs.
7. The DEI Round 5 and Round 6 included 12 grantees.
Round 5 2014–2019
California (n=8,141)
Illinois* (n=3,120 of which 2,560 were youth)
Kansas (n=3,214)
Massachusetts**
Minnesota (n=5,440)
South Dakota (n=694)
Round 6 2015–2019
Alaska (n=310)
Georgia (n=84)
Hawaii**
Iowa (n=8,936)
New York (n=3,888)
Washington**
*Youth grantee/**Grantee did not provide WIA data
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
7
Round 5
California Employment Development Department
# of prior DEI grants: 1= R2
Workforce Services Division Illinois Department of Commerce and Economic Opportunity
# of prior DEI grants: 2= R1, R4
Kansas Department of Commerce
# of prior DEI grants: 1= R1`
Massachusetts Executive Office of Labor and Workforce Development, Department of Career Services
# of prior DEI grants: 1= R3
Minnesota Department of Employment and Economic Development, Division of Workforce
Development
# of prior DEI grants: 1= R3
South Dakota Department of Labor and Regulation, Workforce Training
# of prior DEI grants: 1= R2
Round 6
State of Alaska, Department of Labor and Workforce Development
# of prior DEI grants: 2= R1, R4
Georgia Department of Economic Development, Workforce Division
# of prior DEI grants: 0
State of Hawaii Department of Labor and Industrial Relations
# of prior DEI grants: 1= R2
Iowa Workforce Development
# of prior DEI grants: 1= R3
New York State Department of Labor
# of prior DEI grants: 1= R4
Washington State Employment Security Department
# of prior DEI grants: 1= R2
8. DEI provided coordination of integrated training and support services. DEI training and
support services provided adult and youth JSWDs with integrated workforce development
systems designed to improve training and employment outcomes. Individuals in a treatment
WDA who self-disclosed a disability were eligible to enroll in the DEI. The program did not
require individuals to provide documentation regarding the type or severity of their disability.
DEI participants received one-on-one support from a DRC. DEI participants from 12
grantees self-disclosed a disability but occasionally did not directly discuss their disability
with a DRC.
9. SDSs facilitated JSWD outreach and engagement and employer Partnerships and
Collaborations. Integrated Resource Teams, Blending and Braiding Resources, and
Customized Employment were implemented with selected features of SDSs, including
Active Resource Coordination, Partnerships and Collaborations, Asset Development, Work-
Based Learning, Apprenticeships, Job Shadowing, and Career Pathways. While there were
differences in the fidelity of SDSs, it was apparent that training such as person-centeredness
and systems integration resulted in improvements in service availability and implementation.
We conclude that while grantees may not have implemented each of their SDSs with fidelity,
the net effect of their introduction appears to have improved Partnerships and Collaborations
with employers and stakeholders and access to work incentives such as Trial Work
Period, Impairment-Related Work Expenses, Plan to Achieve Self-Support, and Partnership
Plus.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
8
10. DEI Round 5 and Round 6 grantees implemented six requirements:
i. Selection of either a youth or adult focus.
ii. Physical and programmatic accessibility of participating AJCs.
iii. Implementation of at least two DEI SDSs: Active Resource Coordination; Asset
Development; Blending and Braiding Resources; Benefits Planning; Customized
Employment; Guideposts for Success; Integrated Resource Teams; Partnerships and
Collaborations; Work-Based Learning Opportunities; and Entrepreneurship.
iv. Completion of a sustainability plan for after the grant period.
v. Availability of CWICs for United States Social Security Administration (SSA)
beneficiaries to provide benefits services.
vi. Implementation of TTW and access to work incentives webinars.
11. SSA Work Incentives Planning and Assistance providers certified CWICs who
provided benefits counseling to beneficiaries seeking employment. Many DRCs were
CWICs who made decisions about the impact of earnings on health care and benefits. Some
SSA beneficiaries enrolled in TTW to limit the effects of working on their continuing
disability review,
3
which can affect health insurance coverage.
12. We identified promising practices among Round 5 and Round 6 grantees. For example,
California’s Traveling DRC provided 1-day trainings for AJC Employment Specialists “to
bring best practices and perspectives discovered through California’s DEI and Disability
Employment Accelerator to improve service delivery to individuals with disabilities in each
treatment WDA.”
4
Illinois DRCs had a strong partnership with Vocational Rehabilitation that
included a dedicated staff member who made referrals to Career Pathways services, enrolled
individuals in WIOA services, and provided oversight of individuals who enrolled in
community colleges. DRCs worked with Vocational Rehabilitation to make referrals to
employment and Career Pathways training while apprenticeships and internships allowed
youth to learn about employment and what it entails (e.g., receiving remuneration, working
in a group environment, adhering to the requirements of a supervisor).
13. While there were no measureable impacts of DEI on wage and employment outcomes,
there were consistently positive impacts of career pathways programs on the same
outcomes. The DEI Round 5 and Round 6 impact evaluation included three distinct research
designs, with the aim of measuring the impact of DEI on employment and wage outcomes, as
well as the career pathways intervention on the same outcomes. Randomization was not used
in the evaluation, but instead, quasi-experimental methods were used to determine the impact
of DEI on outcomes. The evaluation team used administrative that were collected on people
with disabilities who were entering DEI-funded AJCs and comparison group AJCs that were
not funded as part of DEI, and used a matching procedure to help determine the impact of
receiving services at a treatment AJC. The level of comparison was at the WDA level, with
outcomes measured at the individual customer level. The results indicated that although there
3
U.S. Social Security Administration. (n.d.). Understanding Supplemental Security Income continuing disability
reviews 2019 edition. Retrieved from https://www.ssa.gov/ssi/text-cdrs-ussi.htm
4
IL DRC comment, March 9, 2020.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
9
were no sustained impacts of DEI on outcomes, there were consistently positive impacts of
being in a career pathways program on the same outcomes.
II. Introduction
The Office of Disability Employment Policy (ODEP) of the United States Department of
Labor (USDOL) was created in 2001. Its mission is to provide national leadership by developing
and influencing disability employment-related policies and practices affecting an increase in the
employment of people with disabilities. Since then, ODEP has designed numerous programs and
policies that support people with disabilities and the workforce development system by
promoting Career Pathways training and employment, the Campaign for Disability
Employment, and the Workforce Recruitment Program, which connects employers with highly
motivated college students and recent graduates with disabilities. ODEP, in partnership with the
Employment and Training Administration (ETA), has also implemented which focuses on Stay-
at-Work/Return-to-Work strategies. Other projects include the SSI [Supplemental Security
Income] Youth Recipient and Employment Transition Formative Research
Project, which identifies promising programs and policies for youth SSI recipients,
and Evaluating the Accessibility of American Job Centers for People with Disabilities.
A. Disability and Labor Force Engagement
The median income of families living in the United States with at least one member with
a disability is 72 percent of the national median, while median incomes for families with at least
one member with a mental or physical disability are 66 percent and 68 percent, respectively, of
the national median.
5
6
All things being equal, age and disability are the most important
predictors of labor force participation leading to employment, as older workers are more likely to
leave the labor market than younger workers due to the onset of disability or social and economic
factors.
7
These factors include chronic health conditions and the changing social aspects of
income, education, gender, race/ethnicity, and variability in the labor market.
8
9
Labor force engagement tends to decline after the onset of a disability due to chronic
conditions.
10
Chronic conditions have been linked to reductions in household income and
increases in the likelihood of poverty, limitations in human capital, and job skill development.
11
For example, people with cerebral palsy, spinal cord injury, or rheumatoid arthritis have much
higher labor force participant rates than those with visual impairments or intellectual disabilities.
Working-age people with sensory, physical, and/or cognitive disabilities
12
are also at a
5
Minkler, Fuller-Thomson, & Guralnik, 2006.
6
Disability Statistics and Demographics Rehabilitation Research and Training Center. University of New
Hampshire.
7
Mitchell, Adkins, & Kemp, 2006; Rigg, 2005.
8
Mitchell et al., 2006.
9
Crimmins, Reynolds, & Saito, 1999.
10
Mitchell et al., 2006; Young et al., 2002.
11
Loprest & Maag, 2009.
12
Houtenville, Erickson, & Lee, 2012.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
10
significant disadvantage in terms of their employment and earnings when compared with
working-age individuals without a disability. While age is the most important predictor of labor
force participation, there are a variety of health, social, and economic factors involved in
workforce participation, including long-term chronic health conditions, work history, family
income, educational attainment, gender, race/ethnicity, and the vagaries of the labor market.
13
14
III. Description of the Program
The workforce development systems in the United States changed in 2015 with the
transition from the Workforce Investment Act (WIA) to the Workforce Innovation and
Opportunity Act (WIOA). WIOA replaced WIA, which used a hierarchical service delivery
system that required jobseekers to enroll in core services, employment placement, and intensive
services that included skills assessments and enrollment in college. Jobseekers with disabilities
(JSWDs) were required to start from core and intensive services in order to allocate resources
into three categories: disadvantaged adults, dislocated workers, and youth. In 2015, federal
resources were provided to USDOL agencies to WDAs to support the new WIOA programs. In
most WDAs, there is one comprehensive AJC and one or more satellite centers. WIOA provides
three levels of services for adults and dislocated workers: basic, individualized, and follow-up
services that are no longer needed because they can be accessed through WDAs; what were
intensive and training services are now “individualized services” that can be coordinated through
Partnerships and Collaborations.
USDOL’s ETA and ODEP sought to resolve chronic unemployment and limited
opportunities for people with disabilities. (WIOA) increased the capacity of WDAs to serve
jobseekers whether or not they have a disability. Understanding that there is a need for further
investment in JSWDs, the Disability Employment Initiative (DEI) was implemented to increase
the capacity of WDAs
15
and to facilitate Partnerships and Collaborations that improve access to
AJC services for people with disabilities.
16
Funding for DEI Round 5 and Round 6 was provided in 2017 and 2018, respectively,
when ETA and ODEP released a Solicitation for Grant Applications that made provisions for
$14,837,784.70 for Round 5 and $14,911,243.00 for Round 6 to support 12 cooperative
agreements through the Consolidated Appropriations Act of 2010.
17
One-half of the DEI funds
went to ETA for the training and technical assistance (T/TA) of DEI grantees. The other half
went to ODEP for the DEI evaluation.
As with earlier rounds, Round 5 and Round 6 were required to provide services to either
adults or youth participants. The age range for adult JSWDs was 21 or older, while the age range
13
Crimmins, Reynolds, & Saito, 1999.
14
Mitchell, Adkins, & Kemp, 2006; Rigg, 2005.
15
Information about WDAs and their services.
16
For detailed information on promising practices and challenges, go to page 37 for Round 5 and page 103 for
Round 6 grantee narratives. For detailed information on the implementation of DEI SDSs, go to page 15.
17
Pub. L. No. 111-117
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
11
for youth was 14–21. All but one Round 5 and Round 6 grantee did not have a prior DEI grant as
the Georgia Department of Economic Development, Workforce Division, was new to the DEI in
Round 6. Nonetheless, all grantees were positioned to maintain Ticket to Work (TTW) activities,
as many collected resources that provided access to new service delivery strategies (SDSs) and
training prior to project implementation. DEI State Leads, Disability Resource Coordinators
(DRCs), and AJC Employment Specialists helped grantees acclimate to the DEI and recruit
Ticket holders, collaborate with Vocational Rehabilitation (VR), and coordinate Ticket services
under Partnership Plus arrangements.
18
19
Grantees also had access to the following SDSs in
Exhibit 1 to help JSWDs obtain credentials required for in-demand occupations.
Exhibit 1: Complete List of Service Delivery Strategies
1. Active Resource Coordination (Selected Components)
2. Apprenticeships/Job Shadowing (Selected Components)
3. Asset Development Strategies (Selected Components)
4. Career Pathways
5. Customized Employment (Selected Components)
6. Guideposts for Success (Not selected)
7. Entrepreneurship/Self-Employment (Selected Components)
8. Integrated Resource Teams (Selected Components)
9. Leveraging Resources and Services/Blending and Braiding Resources (Selected
Components)
10. Work-Based Learning Opportunities (Selected Components)
20
Most grantees used components of SDSs to provide support for JSWDs. For example,
Active Resource Coordination, Integrated Resource Teams, Work-Based Learning,
Apprenticeships, and Career Pathways were commonplace as they facilitated communications
across WDAs and leveraged resources for transportation, training, employment opportunities,
and college enrollment that supported employment goals. Some SDSs were selected in their
entirety. Others were used as components of SDSs. For example, in the boxes in Exhibit 2 below,
SDSs 14 were more likely to be selected in their entirety versus SDSs 5–8, which were used to
augment specific components of an SDS.
18
Goodley, D. (2010). Disabilities studies: An interdisciplinary introduction. London: Sage Publications.
19
As the statutory-level Employment Network (EN), VR agencies could enter into Partnership Plus agreements with
DEI grantees, allowing VR and AJCs to receive outcome payments through case closures. DEI grantees could also
receive milestone payments for follow-up job retention services.
20
King, C. T., Choi, J., & Cerna Rios, A. (2014). Improving services for persons with disabilities under the
Workforce Investment Act and related programs: Challenges, opportunities, and a way forward. Retrieved from
http://raymarshallcenter.org/files/2014/11/Improving-Services-for-Persons-with-Disabilities-under-the-Workforce-
Investment-Act-and-Related-Programs_APPAM.pdf
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
12
Exhibit 2: Utilization of SDSs
1. Active Resource Coordination
2. Integrated Resource Teams
3. Blending and Braiding Resources
4. Career Pathways
5. Customized Employment
6. Guideposts for Success
7. Entrepreneurship
8. Leveraging Resources and Services
Of the 10 SDSs listed in Exhibit 1, we identified eight that used components of other
SDSs (Active Resource Coordination, Apprenticeships/Job Shadowing, Asset Development
Strategies, Career Pathways, Customized Employment, Integrated Resource Teams, Leveraging
Resources and Services, and Work-Based Learning) despite grantees not having formally
selected them for their projects. By scaling down the number of SDSs, grantees implemented
SDS components that allowed them to introduce and scale-up a broader array of support services
for JSWDs. In addition, using facile and familiar approaches to SDS implementation played a
role in accommodating the frequency of in-person meetings between DRCs and JSWDs through
Integrated Resource Teams (augmenting them when necessary with telephone and/or email
meetings).
Each SDS was used to “individualize the employment relationship by identifying the
strengths and interests of each JSWD and utilizing a menu of customized assessments and team
planning that, in optimal situations, involved AJC Employment Specialists and VR counselors.
Furthermore, Round 5 and Round 6 grantees focused on systems change and improving access to
services for JSWDs among multiple partners, including AJCs, and augmenting their systems
change strategies by accessing resources from DEI and T/TA providers. Specific components of
Customized Employment and Work-Based Learning, including on-the-job training (OJT) and
internships, were available to help JSWDs obtain credentials required for in-demand
occupations. SDSs were implemented concurrently and often in partnership with other entities
such as USDOL Jobs for Veterans State Grants staff and local industries and employees. DRCs
and related partners were also involved in systems change, aiding WDA staff and providing
services for JSWDs designed to improve staff skills.
Due to their experience with prior DEI projects, Round 5 and Round 6 grantees did not
have many of the same difficulties of earlier grantees, such as the implementation of TTW.
While Alaska (Round 6) had some challenges throughout their grant period with SSA program
requirements, other grantees mentioned resource limitations and limited access to Community
Work Incentives Coordinator (CWIC) services, despite a less lengthy SSA suitability
determination process as compared to DEI Round 1 through Round 4 projects, grantees were
able to address these issues, and overcome them. These issues had been resolved due in large
part to the considerable experience of ODEP-ETA, DEI State Leads and DRCs, and an approach
to systems change that integrated employment and training into local communities with the
assistance of DRCs and experienced WDA personnel. DEI knowledge of ways to address
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
13
systemic barriers was also enhanced by DOL’s WorkforceGPS webinars and DEI peer-peer
learning exchanges.
While it is well-known that some JSWDs had trepidations about working due to a fear of
loss of SSI/Social Security Disability Insurance (SSDI) benefits, they generally reported that in
many instances CWICs allayed their concerns. CWICs, many of whom were DRCs, provided
counseling and outreach to beneficiaries who were eligible for work incentive programs. Free
CWIC services ensured that individuals on SSI or SSDI could transition to employment with the
help of SSA work incentives. The creation of ENs also facilitated systems change as DEI State
Leads, DRCs, and SSA staff worked together to implement ENs and create expectations and
working relationships that would leverage TTW resources to support AJC services for JSWDs
after the grants ended.
IV. Evaluation Design
The goal of the DEI was to improve the employment outcomes of JSWDs through the
application of specialized SDSs and the facilitation of systems change. The purpose of the DEI
impact evaluation was to measure and assess the outcomes and impact of DEI Round 5 and
Round 6. The main research questions focused on employment, employment retention, and
wages of DEI participants, while other questions focused on how specific SDSs impacted
participant outcomes. The source of information for the impact analysis came in the form of
WIA administrative data, which had been collected on individuals who received any form of
services at an AJC.
To provide context on the individuals with disabilities who received services outside of
DEI-funded AJCs, a survey on employment, activities of daily living, and disability type was
analyzed and reported in this evaluation. The source of information from this analysis came in
the form of a novel survey that was administered via web and phone to individuals in comparison
group AJCs who elected to answer questions in exchange for a $15 gift card.
The main impact analysis determined the effect of the DEI intervention on outcomes
using two quasi-experimental approaches. In addition, a tertiary analysis determined the effect of
Career Pathways programs on outcomes using only individuals who received services in the
treatment group. To provide context for these findings, the study examined descriptive
characteristics of all the individuals in the WIA data sample as well as findings on activities of
daily living and disability type by individuals in only the comparison group. The overall findings
suggest that although DEI as a program did not have statistically significant impacts on
outcomes, Career Pathways enrollment did have impacts on outcomes even after accounting for
selection bias.
We designed the primary impact analysis to measure differences in program outcomes
using a quasi-experimental design (QED) that compared treatment group to the comparison
group WDAs at the individual per-person level. The primary analysis included only state
grantees that had both treatment and comparison WDAs. A secondary analysis using a short
interrupted time series (SITS) design included treatment WDAs in order to accommodate those
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
14
states that only had a single WDA or had comparison WDAs that may have also received
services funded through DEI.
21
The SITS analysis provided a baseline impact measure on how
outcomes changed before and after the appearance of interventions for each round of grantees.
22
A tertiary analysis used WIA data from only the treatment group individuals to determine
whether Career Pathways programs had an impact on outcomes by delineating two groups:
individuals who enrolled in Career Pathways programs and those who did not. By using a QED
that compared these two groups, we determined if the participation in Career Pathways programs
impacted individual-level employment, employment retention, and earnings and outcomes.
V. Program Implementation
Fundamental to DEI practices were the ways that DRCs mobilized stakeholders and
resources to build strong, supportive services around the jobseeker in ways that respected their
self-direction and choice. Consequently, DEI State Leads and DRCs often found themselves
engaging in systems change through Partnerships and Collaborations and services on a daily
basis. Partnership development also included helping state, local, and community-based agencies
build capacity by training their staff in DEI SDSs. While some DEI grantees focused more on
particular job and career development strategies (e.g., Career Pathways, Customized
Employment, Self-Employment, Asset Development), they all involved some measure of
coordinated, leveraged team support leading to employment through the use of SDSs. Each of
the SDSs utilized by DEI grantees are described below in greater detail.
A. Implementation of DEI Service Delivery Strategies
As mentioned earlier, grantees used components of SDSs to operationalize Active
Resource Coordination and Integrated Resource Teams (IRTs), Work-Based Learning and
Apprenticeships, Career Pathways, youth in transition with disabilities, IRTs that facilitated
communications across WDAs, and leveraging resources for transportation, training,
employment opportunities, and college classes that supported employment goals.
21
California and Illinois were deemed to have comparison group WDAs that may have received services or benefits
from DEI funding and were excluded from the QED analysis. South Dakota (Round 5) and Alaska (Round 6) each
had only one WDA and were also excluded from the QED analysis. Massachusetts (Round 5), Hawaii (Round 6),
and Washington State (Round 6) did not make their data available for analysis.
22
It should be noted that quasi-experimental approaches cannot fully eliminate selection bias, and that proper
identification of the intervention effect is more difficult to determine than a randomized trial at the individual level.
Identification is particularly difficult to accomplish at the WDA level, when we know that some comparison group
WDAs were offering services that resembled the ones in some treatment group WDAs. Although the matching
strategy helps to make sure that individuals across treatment and comparison WDAs are similar to one another, the
authors acknowledge the difficulty in completely isolating the effect of DEI on outcomes. The authors have taken
states out of the impact analysis completely when there is evidence that there is an abundance of contamination in
the comparison group (e.g., California for the matched comparison group QED).
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
15
The figures and text below illustrate how various SDSs optimally flowed from the point
of outreach and engagement to employment and supportive services. Improvements in support
services were achieved when services were delivered with substantiated principles and practices.
Despite the fact that over the course of the DEI Round 5 and Round 6 grants DRCs and partners
had access to extensive training on SDSs, implementing them with fidelity was inconsistent.
While we found dispersion regarding the fidelity of SDSs, it was also apparent that
training focusing on person-centeredness and systems integration aspects inherent to all DEI
SDSs at the very least resulted in improvements in attitudes, awareness, and service availability
for JSWDs being served at AJCs compared to other comparison WDAs.
“Generally, we conclude that even though many grantees did not implement each SDS with fidelity, the
overall net effect of their introduction and use appears to have improved their partner agencies’
employment services by providing access to SDSs and learning how they may contribute to training and
employment. The following provides descriptions of each SDS implemented by DEI round and state.
Active Resource Coordination (ARC)
Round 6 (R6)
Alaska AK
Hawaii HI
Iowa IA
New York NY
Washington – WA
Kansas: ARC occurred regularly statewide. DRCs frequently engaged other staff from
the AJC and other partners with resources because “we want everyone to get all the resources we
have.” We also heard that without DEI, ascertaining this information and resources would likely
be “pushed back to the participant.” One partner agency indicated that “more regular meetings
could improve ARCs, alluding to the fact that sometimes it was a difficult task to get all
necessary partners to the table to better understand the resources they could collectively bring to
bear for a particular individual and ARC. It was described by a DRC as “finding the best
resources to meet individuals’ needs.”
South Dakota: ARC was practiced in Rapid City before enrollment as the first step of
forming an IRT, as is appropriate to the definition. It involved discussing eligibility and
employment goals. There was also statewide training on Active Resource Coordination.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
16
Asset Development
Round 5 (R5)
Round 6 (R6)
Not selected by any
R5 grantees
Washington – WA
Hawaii HI
Generally, when queried about Asset Development, most respondents appeared to equate
that service with Benefits Planning. However, even though they are aligned, each is a distinct
SDS. We obtained data by interviewing DEI leaders, DRCs, and partners and stakeholders to
determine how extensively a full range of Asset Development services was being provided, or if
grantees indicating that they focused on Asset Development were only providing benefits and
work incentives planning, even if only basic personal budgeting. In this regard, the correlation
between Asset Development and Benefits Planning is strong because DEI customers who
received SSI/SSDI and/or veterans’ benefits or other forms of public assistance needed to know
how to manage their earnings from work and how they could spend and save money without
jeopardizing their benefits.
Although Alaska, Hawaii, Washington, and South Dakota selected Asset Development,
of these states, Alaska, Washington, and South Dakota made significant progress in creating and
strengthening Asset Development partnerships.
Alaska: DRCs often worked at an individual level in terms of Asset Development by
giving one-to-one advice on benefits and budgeting. One WDA indicated that a section on
budgeting was included as part of the basic training program. Alaska required all DRCs to go
through CWIC training for Benefits Planning. Frequently, some aspects of financial literacy were
provided, as one DRC from Alaska covered budgeting after her clients got full-time jobs and
advised them to be cautious of scams. She said, “Just be aware, watch your social media. Then
budgeting and saving. I do address that.And another DRC integrated some aspects of financial
planning into her Benefits Planning work with customers by referring them to marriage and
family therapy and resources from the Volunteer Income Tax Assistance service.
New York: All DRCs were trained to answer questions related to Asset Development
and become familiar with the needs of their AJCs in this regard. Some DRCs attended a monthly
CA$H Coalition meeting and promoted both Volunteer Income Tax Assistance at the Albany
AJC.
I’m a member of the CA$H Coalition. It’s part of United Way here. It’s creating assets, savings,
and hope. It’s a group that meets monthly and includes an SSA rep, which is fabulous. We get
together and do outreach, attend some political kick-off type things, on behalf of our customers.
DRC
South Dakota: South Dakota integrated Asset Development strategies that promoted
financial literacy, credit counseling, and tax assistance. There was also evidence of helping
JSWDs with public assistance related to housing, child care, and transportation. A number of
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
17
examples substantiate how the DEI was networking with financial literacy resources on behalf of
its customers. One WDA hosted a Wells Fargo Financial Literacy class in conjunction with a Job
Search Assistance class and My Free Taxes information that was shared at the South Dakota
Youth Council meeting. The WDA also hosted two Credit When Credit is Due classes, taught by
a local partner of Consumer Credit Counseling of the Black Hills. We also found some evidence
that local AJCs offered financial literacy classes to participants.
Washington: Washington understood the importance of integrating Asset Development
and financial literacy into its services strategy. From the outset, they developed their partnership
with the Northwest Area Foundation to provide financial literacy and income stability training
and support to participants. While our data substantiates that a number of DEI grantees included
Asset Development and financial literacy along with Social Security Benefits Planning,
Washington took a statewide approach to the integration of these services by leveraging partner
resources for ongoing access to expertise and training and provided a substantial number of
JSWDs with financial wellness training sessions over the life of the grant.
Blending and Braiding Resources
Round 5 (R5)
Round 6 (R6)
California CA
Alaska AK
Illinois IL
Georgia GA
Kansas KS
Hawaii HI
Minnesota – MN
Iowa IA
South Dakota – SD
New York NY
Washington – WA
Blending and Braiding Resources was frequently observed across Round 5 and 6 DEI
grantees. It is a process that is integral to ARC and IRTs and is an essential component of
systems change. Examples of Blending and Braiding Resources are provided below.
Alaska: Alaska embedded Blending and Braiding Resources quite effectively due to its
strong partnerships and the use of IRTs to support youth with disabilities in exploring various
career paths and educational opportunities. One DRC gave an example of a youth customer who
needed additional help with housing and transportation. According to the DRC, “I worked with a
bunch of different agencies on that. We were able to get him into Job Corps. … And they got him
in, right at the end of May, which was really nice because that’s when his housing was over and
then in Alaska, to do that temporary housing would have been so expensive.” The collaboration
between DEI and Department of Vocational Rehabilitation’s (DVR) Pre-Employment Transition
Services program resulted in co-funded summer youth employment programs, institutes for
youth in the criminal justice system, and shared financial support for training, transportation, and
other youth employment-related services.
California: California Blended and Braided Resources both at the state and WDA levels.
At the state level, the grantee combined resources from DEI and the Disability Employment
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
18
Accelerator (DEA). While working with multiple DEI projects, the DEI State Lead described
how it gave the state an advantage to blend resources. The availability of such resources helped
at the AJC level — in California, the state provides 50 percent of the OJT. One of the DRCs in
the state noted a few success stories of the clients going through OJT and on to permanent
employment. Furthermore, the strong partnership contributed to a more efficient blending and
braiding approachthe regional centers they worked with provided respite care for individuals
with disabilities and mental health, as well as training for ABLE accounts. As a result, the DRC
learned that establishing and maintaining partnerships was a key to blending and leveraging
resources. The DRC prepared a Disability Resource Guide, which listed the partner agencies
they worked with, contact information, and types of services they provided.
Kansas: Kansas Round 5 used Blending and Braiding Resources. One DRC stated that
he was “putting a greater emphasis on making sure that individuals are enrolled appropriately
for system-wide support. When I took over, there were many individuals that we were servicing
through WIOA that had not been co-enrolled in DEI.” Kansas also had strong collaborations
with partner agencies and DRCs who utilized partnerships in Blending and Braiding Resources.
DRCs often “pick up support services so Blending and Braiding of funding, Active Resource
Coordination, if you're talking about finding the best resources to meet the individuals’ needs,
that’s occurring on a regular basis.In another Kansas WDA, co-enrollment was a strong factor
in Blending and Braiding Resources. According to one DRC, “We always co-enroll with DEI.
We also co-enroll with our adult program, our youth program, whatever they fit into. We try and
fit as many co-enrollments as we can, because it serves people better.
Illinois: One AJC had good success with Blending and Braiding Resources. This AJC ran
into an issue serving youth, as their WIOA Youth program at the AJC could serve very limited
numbers of in-school youth; the WDA resolved this problem by having access to DEI funds in
the high school used to serve DEI-eligible students, while those who were out of school could be
served at the AJC using WIOA funds.
Massachusetts: The DEI grantee partnered with the Massachusetts Rehabilitation
Commission and the Department of Developmental Services to leverage resources in order to
provide extended services to JSWDs. These partners also helped pay for transportation and
provided job coaching for individuals once they started work. AJCs also leveraged DVR
resources to pay for transportation, but Blending and Braiding of other sources of funding was
limited. The grantee viewed Blending and Braiding Resources as a best practice, as one
respondent noted that it was very useful in getting a client everything they needed, such as
clothing or a driver’s license, because different agencies could pay for different services.
South Dakota: According to the State Lead, “We have tons of Blending and Braiding
examples. Someone might provide clothing for a client who needs them in order to work, or some
other needed resources. An example was a man that needed special clothing in an extra-large
size and that meant having them tailored. We split the bill with another agency. They also split
costs for training and transportation. Blending and Braiding was deemed essential to successful
IRT implementation in South Dakota, and they indicated that it was very likely to continue after
the grant ended because agency partnerships had been so well established.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
19
Washington: DRCs focused on Blending and Braiding Resources. The DRC used WIOA
or Basic Food Employment and Training services for school enrollment and tuition. These Basic
Food funds paid for tuition, books, tools, test fees, etc. However, there seemed to be room for
more active Blending and Braiding at this site because a customer noted that they still had
trouble with their housing. As a DRC said,The big challenge would be housing. That’s the
whole issue. DEI is not helping with housing at all.” In another WDA, the process for Blending
and Braiding seemed to be more stable. The DRC tried to utilize WIOA funds as much as
possible. Additionally, when a customer’s needs were not covered by WIOA, the DRC tried to
find additional resources for support. For instance, a customer noted that he received interview
clothes because those were not funded by WIOA, while the other customers were directed to
Goodwill for additional support.
Career Pathways (CP)
Round 6
Alaska AK
Hawaii HI
Iowa IA
New York NY
Washington – WA
Illinois: Illinois made CP a major part of its service delivery strategy. One AJC
developed close partnerships with its local community colleges and worked together to serve
JSWDs. The AJC and community colleges referred to each other, and the colleges worked to
find the best CP for each JSWD and offered job readiness courses and internships. The
community colleges also did outreach to high school students and attempted to get them involved
in employment and training. In addition, a local high school system made exposure to different
career options a central part of its curriculum. Students could select a career-focused track that
included an internship in a related career through coursework in high school and at a local
community college.
Kansas: Kansas leveraged its DEI funds to support longer participation in credentialing
and training programs that were more conducive to stackable credentials and career ladders.
According to the DEI State Lead, the DEI was instrumental in formalizing relationships with
postsecondary institutions and enhancing accommodation policies at local schools. Those who
participated acquired and maintained employment for at least 6 months at a higher average
hourly wage than projected.
Minnesota: A particularly robust employer engagement strategy was implemented in
three treatment WDAs focused on partnering with employers to provide CP-focused
credentialing, education, and apprenticeships in the career sectors of manufacturing; health care,
including certified nursing assistant (CNA), cardiopulmonary resuscitation, automated external
defibrillator, home health aides, personal care assistance, and medical technology; office and
administrative technology; precision sheet metal; retail management, customer service, and sales;
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
20
welding; food “ServSafe;” and commercial drivers. In addition, the involvement of JSWDs in
CP-focused training and employment was strengthened by opportunities to participate in peer
mentor meetings where jobseekers could both obtain and provide peer support to help offset any
challenges they encountered. Minnesota’s CP featured a close integration between acquiring the
credentials needed in a pathway and preparation for securing a job.
New York: WDAs developed health care pathways, including a certified nursing
certificate, while other participants worked toward a licensed practical nurse credential.
Manufacturing was another pathway that New York identified. DRCs engaged as advisors and
facilitators in crafting CP approaches and braiding resources with DVR, community colleges,
and community-based agencies. DRCs also leveraged resources from Pell grants.
South Dakota: South Dakota employment infrastructure was buttressed by its proximity
to the region’s technical school and leading adult education/supportive services provider. It also
took the lead in producing and disseminating the Building Pathways toolkit and CP visuals to
guide JSWDs with documentation on Job Shadowing for staff, clients, and employers. Statewide
trainings incorporated best practices around securing accommodations for individuals with
disabilities to complete a General Education Diploma (GED). Together, South Dakota, the Board
of Regents, and the South Dakota Department of Education built a Career Pathways development
web tool to be used by multiple partners.
The Career Navigator goes out to the college and meets with the students we have and talks
with the instructor and student in terms of how they’re doing. And then when they start their job
search they meet with Career Navigators and do enhanced résumé writing to really tailor-make
it to that specific job/skill set.” — AJC Staff
Round 5 and Round 6 grantees appeared to scale-down the number of SDSs they used
relative to earlier rounds. In prior grants, grantees were required to select two SDSs. Logistical
challenges, however, may have led to more facile and familiar approaches to using SDSs. It is
striking that two Round 5 and Round 6 DEI grantees (Georgia and Hawaii) selected Customized
Employment but did not implement it in its entirety. Guideposts for Success, IRTs, Work-Based
Learning Opportunities, and Self-Employment were other previously favored SDSs. Components
of each of these SDSs were used by Round 5 and Round 6 on a less regular, informal basis by
individualizing the employment relationship through job carving; better understanding the
interests, strengths, and needs of each JSWD; and the use of Customized Employment’s
customization and negotiation strategies.
Social Dynamics researchers identified DRCs who
implemented IRTs, albeit in less coordinated ways than we saw in earlier rounds. DRCs
speculated that IRTs would continue to be implemented with less formality by Round 5 and
Round 6 grantees, as fewer incorporated the coordination of multiple services and leveraged
resources arranged by the DRC in support of the jobseeker (Exhibit 3).
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
21
Exhibit 3: ODEP-Suggested DEI Round 5 and Round 6 Service Delivery Strategies
Customized Employment (CE)
Round 5 (R5)
Round 6 (R6)
Not selected by any R5
grantees
Georgia GA
Hawaii HI
We obtained data that indicated an uncertain level of understanding and use of CE among
DEI grantees. However, to get a more complete picture of how CE elements were implemented
by DEI grantees even where not specifically chosen, we scanned interview and progress report
data and found that some of those grantees were in fact reporting the use of CE practices and
labeling person-centered employment planning as “Customized Employment.”
We found no instances of model fidelity that included CE implementation from
Discovery through to customized jobs. However, we did find adoption of some CE elements,
particularly Discovery, by a number of DEI grantees. Customized Support Teams are an integral
part of the CE model, and IRTs appear to offer a close approximation of CE team development.
Where IRTs occurred, it is reasonable to assume that they aligned with the principles, if not all
the practices associated with Customized Support Teams. DEI grantees also mentioned that
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
22
Discovery and CE aligned well with WIOA Title I and DVR services as enhancements to
person-centered assessments.
23
We view the CE-related elements that DEI grantees were implementing as a promising
practice because of the potential ways that they can be embedded within and reimbursed by DVR
and WIOA services systems. As staff used these methods, they appeared to reduce the need for
JSWDs to “impulse shop” for jobs.
Georgia and Hawaii stated in their grant applications that they intended to implement CE.
Other states were found to have implemented some strategies that aligned with or were based
upon CE. These are included below.
Georgia: The DEI State Lead and DRCs were trained in career mapping, a person-driven
assessment process that shares many elements with CE and Discovery.
We do career mapping with everyone, which is a Discovery tool. We do that usually before we
enroll in Title 1. It’s before a DVR referral. The DRCs are good at that. After that, you do a
resource planning meeting where you identify needs. DRC
Georgia selected and adapted elements of CE. DRCs reported that there was improved
flexibility in providing services to customers based upon their needs and preferences, although it
appeared to be driven by an impulse to provide good customer service, not guided by use of a
formal CE methodology. One exception was due to an employer’s interest in adapting the
conditions of employment in order to hire and retain people with disabilities. However, these
actions were not guided by the formal components of CE, such as carving, negotiation, or job
creation, but bore a greater similarity to the use of flex work, which many employers offer.
Under flexible work arrangements, employees can change the place where their work is done —
for example, by working from home or from a mobile location — or the hours when their work is
done, with different work start and end times, job sharing, or flexible or compressed workweek
schedules.DEI DRC
Hawaii: Hawaii intended to focus on CE as one of its SDSs. Its T/TA partner, the
University of Hawaii, Center on Disability Services, provided numerous orientations to CE
principles and practices for DRCs. However, significant DRC turnover, due in part to a change in
the WIOA contractor; the part-time allocation of DRCs up until the contractor changed and
increased new DRCs’ time on the project; and changes in DEI leadership all contributed to
uneven CE knowledge translation and significantly curtailed its use. Consequently, little
evidence was found that Hawaii implemented CE as it proposed to do.
Kansas: Although Kansas did not select CE as a service implementation priority, the
state had received prior training on its implementation. CE appeared to be implemented with
mixed results. One DRC reported that they had success in adopting the CE Guided Group
Discovery technique and embedding it into their participant assessment and career development
23
Workforce Innovation and Opportunity Act information.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
23
process. However, some DRCs had trepidation about the match between Discovery and what
AJCs were required to provide because of the time it took and the potential effect that had on
achieving their performance outcomes.
Discovery is kind of hard for us in the workforce center. Really, Discovery is not a concept
that’s going to be that easily useful for us.” — DRC
One DRC described a customized job that she had created, but the JSWD did not perform
well in their position. Instead of letting him go, the employer created a position for him at
another location in the business that was a better fit. This is an example of job customization
without labeling the process as CE.
They were able to manipulate an active, open position and change the job description to fit his
ability and his needs.” — DRC
Minnesota: Minnesota also did not select CE as one of its focus areas. Minnesota
received CE training in the past, and some aspects of CE were implemented for some
individuals, although not across all DEI treatment WDAs.
There has been pretty intensive and very helpful training on CE at all three sites over the years,
but CE is being implemented in components, piecemeal with some efforts at job carving and
job sharing, Discovery, on-the-job training, Job Shadowing, … and flexible scheduling.
DRC
California: Although not focused on CE within their Round 5 DEI grant, some elements
associated with CE were used to strengthen a person-centered approach to employment
development.
All three sites are doing a great job of highly individualized placement, really paying attention
to where the individual’s interest and abilities lie to finding them work that will set them on a
good retention and career path, but not so much carving out or customizing employment. The
DRC has done a lot with the Discovery assessment. That interview style of asking ‘why does this
interest you’ is important.” — DRC
New York: Although CE was not formally implemented, the methodology for working
with customers suggested a more comprehensive, informal assessment process that was
conceptually aligned with CE principles. The DRC would not call it “Discovery,” but talked at
length about getting to know each person more holistically rather than working through formal
assessments and standardized intake forms that Discovery is intended to improve upon.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
24
Guideposts for Success
Round 5 (R5)
Round 6 (R6)
Illinois IL
Alaska AK
Georgia GA
We obtained data that indicated the level of understanding and use of Guideposts for
Success among Round 5 and Round 6 DEI youth-serving states. Only Illinois selected a focus on
youth with disabilities in Round 5, and only Alaska and Georgia focused upon this population
among Round 6 sites, although Hawaii shifted its Round 6 focus to an emphasis on youth in
2017. The following examples are representative of Guideposts implementation.
Alaska: Guideposts for Success appeared to be an important component of the Alaska
DEI when developing comprehensive service plans for youth. DEI leadership described how
Guideposts were central to the development of employment goals: “When our DRC1s and
DRC2s are working with our youth and they are creating their plans, they’re going to the
Guideposts. So those are their goals. They are creating their goals from their Guideposts, from
the six strategies.” DRC
Illinois: Although the grantee did not report using Guideposts for Success as an SDS, a
DEI-funded partner agency did make some use of the Guideposts. This partner received training
from a TA provider on implementing the Guideposts and used them as a guide when producing
materials to help serve students with disabilities. This partner reported that they viewed the
Guideposts as a “road map,” a useful tool that they wanted to continue to use after the grant
ended to help produce materials for students with and without disabilities.
Integrated Resource Teams (IRTs)
Round 5 (R5)
Round 6 (R6)
California CA
Alaska AK
Illinois IL
Georgia GA
Kansas KS
Hawaii HI
Massachusetts MA
Iowa IA
Minnesota – MN
New York NY
South Dakota – SD
Washington – WA
All Round 5 and Round 6 states were required to implement IRTs. Consequently, IRTs,
along with Career Pathways, were among the most commonly identified and understood DEI
SDSs. We found IRTs to be among the most prevalent DEI SDSs, and they were described by
almost all grantees as being among the most useful SDSs in creating comprehensive, customer-
centered employment plans. DRCs were often able to describe the function and intent of an IRT
and how they were implementing them without specifically labeling the process as an IRT.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
25
Alaska: DRCs in Alaska viewed the IRT as a key SDS to leverage resources to ensure
that individuals entered a career path that was the right fit with all necessary supports available
for success. IRTs could involve multiple players. In one WDA, an IRT was comprised of the
Office of Public Advocacy, Juneau Alliance for Mental Health Inc., DVR, and Job-X. The DRC
reported that as a result of a strong IRT, a customer was accepted into the Carpentry Union, and
the IRT was still working on trying to obtain more Work-Based Learning experiences to build
credentials for job entry.
California: Based on strong partnerships developed by each local area, the DRCs were
more readily able to hold IRTs focused on the customer that engaged multiple stakeholders in
ways that they had not done before. DEI staff at all pilot sites stated that they used IRTs often
and found them useful.
One of the best things about IRTs is that, if done correctly, which all three sites are [doing],
they are really coming together around one plan. And that’s the difference between consecutive
versus sequential services. Prior to the IRT, you’d end up with a jobseeker who might be working
with multiple agencies, but each agency could have a slightly different employment plan, which
pulls the jobseeker in a variety of different directions. With the IRT, having one plan really
creates a focus and it gets the individual, at least I’ve seen, employed more quickly because
everyone’s on the same page.DRC
Another feature of IRTs is relevant to youth JSWDs due to its value in engaging their
parents:
Bringing the parent on to the IRT and having them see the service providers working together
at the direction of their adult child has really been helpful to negate that fear that the parents
have. That’s been a great benefit with the IRT.” — DRC
Illinois: Respondents at one AJC pilot site indicated that they made use of IRTs
consistently with almost all their customers, and both the AJC and its major partners reported the
IRTs as being a useful tool to get customers their needed resources. However, staff at another
site reported not finding IRTs necessary for all customers, although they were useful in engaging
and working with VR.
Kansas: All WDAs actively used IRTs. One DRC mentioned that scheduling and
interacting with partner agencies on behalf of the customer was a daily task. The IRT often
involved partner agencies, VR, business services, and WIOA case managers. Kansas emphasized
the importance of sitting down at the table with WIOA partners to “figure out what’s going on.
Maybe you have an employer I didn’t think about, maybe you’ve got an idea that I didn’t think
about, and vice versa.” At the other location, the DRC was also using IRTs frequently to bring
multiple perspectives focusing on the needs of the customers.
Massachusetts: DRCs managed and coordinated IRTs by building partnerships and
bringing resources to the table. One WDA seemed to have a relatively strong capacity to manage
and coordinate IRTs by leveraging various partnerships necessary to providing a holistic
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
26
approach to serving customers. Although a DRC at one pilot site discussed how they formed
IRTs through leveraging resources, they noted that formal IRTs were rare. While the essential
IRT functions were in place in Massachusetts, the IRT as implemented did not adhere to all
components of the practice.
New York: A DRC in New York noted that they used IRTs on regular basis, especially
when the client needed extensive assistance. In one of the pilots, the DRC noted that the
employment counselors were often present during the IRT with the client. She also noted that the
AJC tried to train staff to spot the individuals who might need an IRT at an early stage to prevent
the clients from going through unemployment for a prolonged period.
South Dakota: South Dakota implemented IRTs and described them as core to their
services on behalf of JSWDs. But, in addition to individual, customer-centered IRTs, they also
developed “agency IRTs,” which were held with the WIOA core programs, United Sioux Tribes
who operated a Native American program, and a local transit agency and can be considered a
promising practice. “We used IRTs in all three sites we also embraced this strategy and did
trainings across the state with core and other agencies. The IRT model is going well as a
management technique. We had a DVR supervisor in Rapid City ask, ‘Why don’t we do this for
agencies and collaborate with community groups, too?’While IRTs can take time and be
intensive based upon a person’s circumstances, the DEI State Lead reported that because AJC
staff were achieving better employment outcomes with IRTs than they had in the past, they
believed that it was worth the effort.
Washington: Both WDAs in Washington offered that the IRT was a crucial part of their
DEI. Similar to data obtained from other states, IRT-like functions were being implemented
without labeling them as IRTs. As one DRC stated, “we don’t use the term but we do that all the
time because it’s an interdisciplinary team. We work with providers and a community rehab
provider for job placement, case managers, families, etc.Also, customers noted in the focus
groups that the DRC was helpful in finding the right services and connecting them to the right
personnel using a process that included identifying their needs, employment history, and
experience, and eventually forming an IRT with partner agencies that provided their particularly
appropriate services.
Work-Based Learning (WBL) Opportunities
Round 5 (R5)
Round 6 (R6)
California CA
Alaska AK
Illinois IL
Georgia GA
Kansas KS
Hawaii HI
Minnesota – MN
Iowa IA
South Dakota – SD
New York NY
Washington – WA
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
27
Work-Based Learning was most often integrated with other DEI SDSs, including the
experiential aspects of goal-setting within CE, Discovery, and/or CP. The states below offer a
glimpse into how they integrated WBL into their overall menu of services.
Alaska: The partnership between DEI, VR, and other provider agency partners to
implement summer employment opportunities for youth was an example of integrating WBL
with career exploration, goal-setting, and soft skills development. With DEI, DVR, and other
braided resources, foster care youth from around the state were flown into a “transition to
employmentexperience in Anchorage, where they focused on job goal development, workplace
environment soft skills, actual work experiences, and life-skills training that provided real-
time/real-life employment experiences for youth with disabilities.
Massachusetts: One individual worked at a manufacturing company, but she recently
started working in an internship at a local hospital that was planning to offer her a full-time job at
the end of the internship. The local VR agency also partnered with a DRC to offer OJT to DEI
clients as a component of assisting JSWDs to obtain and advance in their careers.
Self-Employment
Round 5 (R5) Round 6 (R6)
Not selected by any R5
grantees
Not selected by any R6
grantees
Self-Employment was not selected as an area of focus by any Round 5 or 6 grantees.
However, it did tend to emerge as a topic of discussion during interviews, particularly in states
where DEI grantees served rural areas and where a relative dearth of jobs existed, but DRCs
were not specific about how they assisted prospective entrepreneurs with disabilities. Overall,
other than Alaska, little evidence supports the use of Self-Employment as a DEI SDS by Round
5 and Round 6 grantees.
Alaska: DEI staff and partners regarded Self-Employment as a viable employment
strategy for JSWDs in areas where few opportunities for wage employment existed. DEI
leadership and their partners, particularly DVR, helped JSWDs access Self-Employment training
and business development resources by leveraging support among their partners. In Anchorage,
they developed the Alaska Business Week magazine and dozens of employers participated as
mentors. Youth were brought in from across the state for a weeklong business development
camp and they began from the ground up to start a business.
Self-Employment? We actually do have it because there is a population of students who live
at home and support their family. That is something unique about our region. We have
business and marketing Entrepreneurship. We bring in artists, and students learn from
resident experts, and we blend that training with business and marketing so they can market
their arts or carving. DEI Partner
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
28
B. Findings from the Social Dynamics Systems Change Coding Rubric
Social Dynamics used the same systems change analysis for the DEI Round 1 through
Round 4 and Round 5 and Round 6 systems change analyses. Social Dynamics examined how
each grantee used SDSs, the level of DEI stakeholder engagement, and the extent to which that
engagement linked JSWDs to employment-related services and skills. Qualitative data analyses
were central to the Round 5 and Round 6 evaluation, as they were in previous rounds. The
implementation evaluation provided information on the roles, responsibilities, and relationships
of grant-funded positions (e.g., DEI State Lead, DRC) and AJC and WIOA-mandated partners
that participated. We conducted site visits and follow-up telephone interviews on an annual
basis. We also prepared site visit agendas, taking into consideration reading materials produced
by each grantee. Conference calls were held with stakeholders to learn their perspective on
progress made. To ensure confidentiality, we did not attribute observations and comments to
specific individuals nor did we reference their names, titles, or organizational affiliations in this
report.
The evaluation team also trained DEI stakeholders from all 12 grantees on the
requirements of the evaluation, with the principle challenge being the training and retraining of
DEI State Leads and DRCs on the project’s reporting requirements and evaluation. Social
Dynamics created state-specific “binders for each grantee and WDA. The binders included fact
sheets outlining basic grant information, including participating WDAs, grant type (adult/youth),
selection of SDSs, names, addresses, and maps of all DEI sites; confidentiality and informed
consent protocols; state annual WIOA reports; WDA newsletters; a glossary of federal and state-
specific terms and acronyms; and site visit questions and probes.
All site visits began with an orientation for WDA staff. We conducted interviews
annually with 65 individual respondents across DEI Round 5 and Round 6, including DEI State
Leads, DRCs, AJC managers, business services staff, Local Veterans’ Employment
Representatives, Disabled Veterans’ Outreach Program staff, youth workforce programs and
services staff, and WIOA-mandated partners. We conducted focus groups with JSWDs in all 12
WDAs. We used a purposive sampling methodology to select respondents while on site at each
WDA. Site visits were conducted by teams of three trained researchers.
24
The research team
relied on the opinions of “primary respondents,” which included DEI State Leads and DRCs who
were responsible for the implementation of the DEI in each WDA.
We used qualitative analysis to analyze the data from interviews and focus groups. It
incorporates “triangulation,” which involves the use of multiple data sources to produce a
thorough understanding of project implementation. This component of the evaluation focused on
modifications in the way the DEI State Leads and DRCs functioned, how they developed their
skill sets, how SDSs were implemented, and how their use evolved over time. WIOA-mandated
and non-mandated partners, communications with employers, and collection of information on
the operation of TTW were other key topics of inquiry.
24
Researchers also collected “artifacts” such as grantee quarterly reports and written descriptions of procedures or
activities observed at WDAs.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
29
C. Developing Domains and Indicators for Systems Change
DEI systems change represents an adjustment in the way WDAs coordinate and allocate
resources and sustain promising practices after the grant period. Within the context of the DEI,
systems change is a corollary of numerous federal, state, and local initiatives that address the
needs of individuals who are members of disadvantaged and/or disenfranchised groups. Systems
change is needed when improvements in the social, civic, and/or economic circumstances across
WDAs are made more difficult because of a combination of prevailing attitudes, knowledge,
skills, and/or resources that inhibit systems change.
To better understand the dynamics of systems change, Social Dynamics developed the
Systems Change Coding Scheme (SCCS), which was revised in 2015. The SCCS measures the
implementation of DEI requirements, including the implementation of SDSs and the capacity of
each WDA to offer JSWDs employment and related services, including CP training; remedial
classes that relate to the requirements of each Career Pathway were also available to JSWDs.
Definitions of what constitutes systems change vary. However, for our purposes, we
referenced ODEP’s Criteria for Performance Excellence in Employment First State Systems
Change & Provider Transformation.
25
Though the focus of the Criteria is more related to the
developmental disability and VR systems, its relevance in addressing changes to complex
systems for individuals with disabilities correlates with the goal of the DEI. The facilitators of
cross-systems change include the development or refinement of promising practices,
advancements in the legal/policy landscape as they relate to accessibility, accommodations for
JSWDs, and strategies that maximize efficiencies through goal alignment, resource coordination,
and improvements in program performance.
The SCCS is a conceptual framework designed by Social Dynamics. It includes eight
domains and indicators that operationalize systems change as it relates to individual WDAs. The
coding methodology is enhanced by information from interviews and focus groups, where
nuances can be observed that lend greater insight into the challenges inherent in any type of
innovative, large-scale initiative. For example, although an AJC might have had assistive
technology (AT) equipment, observing that it was not easily accessible, or that staff who knew
how to use it were not present, or that staff had limited availability or knowledge of the
technology could corroborate why a DRC may have had difficulties providing universally
accessible services to JSWDs. Therefore, the objective is to achieve reliability and validity in
analyzing the impact of systems change by analyzing data from interviews and observations of
WDA operations using inter-rater reliability with three raters.
The SCCS coding methodology includes a four-point Program Implementation Rubric on
the y-axis and a Program Maturation Rubric on the x-axis. Reliability is achieved when three
coders analyze the same data and produce quantifiably consistent results. To achieve reliable
25
U.S. Department of Labor, Office of Disability Employment Policy & LEAD Center. (n.d.). Criteria for
performance excellence in Employment First state systems change & provider transformation. Retrieved from
http://www.leadcenter.org/system/files/resource/downloadable_version/Employment_First_Technical_Brief__3_0.
pdf
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
30
coding, we use mutually exclusive and clearly defined coding categories. To achieve validity, we
combine discrete items into broader, clearly defined constructs. Our coding methodology focuses
on developing a reliable and valid assessment of implementation and maturation or change. This
process includes decision rules, a codebook, and an internal review to confirm that the data
collection protocol, field experience, and observations are reliable.
System Change Indicators
The Program Implementation Rubric is a four-level ordinal scale used to determine the extent to
which grantees monitor and implement the requirements of the DEI grant (y-axis).
1
2
3
4
No evidence that this
indicator is being
met
Some evidence that an
effort is underway to
implement this
indicator
Evidence that
implementation of this
indicator is partially in
place
Evidence that this
indicator has been
fully implemented
The Program Maturation Rubric is a four-level ordinal scale designed to determine the extent to
which DEI grant activities achieves sustainability of DEI practices (x-axis):
Start-Up (1)
Implementation (2)
Operational (3)
Sustainable (4)
An element in the
earliest planning
phases, not yet
formally
implemented
An element that has been
initially implemented, but
not yet formalized beyond
a trial or experimental
phase
An element that has
been consistently
implemented, often with
involvement of staff
beyond the DRC
An element that will/
has persisted in the
operations of the AJC/
WDA/state beyond the
grant period
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
31
System Change
Domains
Definitions Key Areas of Measurement
1. Capacity to
achieve
integrated
supported
employment
for JSWDs
Development of functional IRTs and
resources and workforce development
systems that are inclusive and change
perceptions, attitudes, and understanding
of the issues related to disability and
employment. In addition, improvements
in access to different types of resources
and SDSs, designated staff with expertise
supporting JSWDs, policies that facilitate
access to services and employment and
access to WIOA services, an experienced
DEI State Lead and DRCs tasked with
managing/coordinating services.
DEI State Lead involvement
in systems change
DRC involvement in systems
change & jobseeker support
EN/TTW activity
Knowledge of SDSs
A plan for SDS
implementation
2. Coordination
& integration
of services
Coordination of employment services for
JSWDs. Integrated workforce
development systems that provide SDSs
and related support services. Partnerships
and Collaborations that facilitate cross-
agency training, interagency partnerships,
shared resources, employer cooperation
and engagement, and innovative
approaches to Blending and Braiding
Resources.
Partnerships & Collaboration
Blending & Braiding
Resources
IRTs
Shared resources
Employer outreach
Asset Development training
Benefits Planning
3. Customer
choice
Customization of products and services to
each JSWD as they make their own
decisions about training and the
employment process. Customer
involvement is part of the design of
products and services, the use of financial
assistance (e.g., SSA TTW, Medicaid,
Medicare, and VR services), and targeted
training that focuses on the individual’s
requirements and needs.
Customer choice
Services supported by system
Existing subsidies/benefits
used efficiently
Training availability
Financial literacy assistance
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
32
System Change
Domains
Definitions Key Areas of Measurement
4. Employer
support &
employer
partnerships
Employers support the recruitment and
hiring of JSWDs. WDA provides support
for employers in forums to discuss their
hiring needs and job candidate pool,
development of position announcements,
pay scales for employment opportunities,
and apprenticeship opportunities and
other forms of training such as OJT and
Career Pathways.
Facilitate recruitment & hiring
of JSWDs
Opportunities to discuss
hiring needs
Recruitment of SSA
beneficiaries
Support in developing
position announcements
Opportunities for enrollment
in CP, apprenticeships, OJT,
& other supportive
employment opportunities
5. Use of
enhancements
to existing
SDSs
Identifying, developing, and/or adapting
innovative practices and approaches to the
use of IRTs, Customized Employment,
Self-Employment, Guideposts for
Success, Asset Development, and
Partnerships and Collaborations.
IRTs, CE, Self-Employment,
Guideposts for Success,
Asset Development, &
Partnerships &
Collaborations with a DRC
or Employment Specialist
Using SDSs to facilitate the
employment process
6. Dissemination
of effective
practices &
outreach to
disability &
employer
communities
Identifying, developing, and/or adapting
practices to the use of IRTs, Customized
Employment, Self-Employment,
Guideposts for Success, Asset
Development, and Partnerships and
Collaborations.
Knowledge dissemination and transfer of
best practices to employers and WDA
partners through webinars and other
formalized methods of communication to
JSWDs and employers.
IRTs, Integrated Resources,
CE, Self-Employment,
Guideposts for Success,
Asset Development,
Partnerships &
Collaborations are available
to JSWDs with a DRC or
Employment Specialist
using other SDSs to
facilitate the employment
process
Communication strategies
such as webinars, issue
briefs, & in-person forums
targeting key audiences:
adults with disabilities,
youth with disabilities,
federal & state agency
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
33
System Change
Domains
Definitions Key Areas of Measurement
partners, & support service
providers
7. Universal
design for
learning
Provide multiple means of representation.
Offer ways of customizing how
information is used. Make learning more
helpful with multiple representations of
course content.
Provide multiple means for
representation, development,
& dissemination of effective
practices & options for self-
regulation
8. Sustainability
Sustainability achieved through system
members developing access to alternative
sources of funding through interagency
partnerships, grants, and legislation.
Ensuring that TTW ENs and DEI SDSs
continue after the grant period. Policy
development and policy change that leads
to the sustainability of DEI strategies and
activities.
Evidence of plans to sustain
DEI strategies & activities:
formal agreements,
Memoranda of
Understanding (MOUs),
identified sources of
funding, new grants, &
legislation
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
34
Program implementation and program maturation are linked to the systems change
indicators. They provide information on the start-up, implementation, operation, and
sustainability of each DEI grantee.
Program Implementation and Program Maturation
Program Maturation
Rubric
x-axis
Program Implementation Rubric
y-axis
Not
Implemented
(1)
Exploring
(2)
Partially
Implemented
(3)
Fully
Implemented
(4)
No evidence
that this
indicator is
being met
Some evidence
that an effort is
underway to
implement this
indicator
Evidence that
implementation
of this indicator is
partially in place
Evidence that this
indicator has been
fully implemented
(1) Start-Up
Grant-funded positions (e.g., DEI State Lead & DRCs) are in place
DEI State Leads are trained to administer the grant & oversee its
implementation
DRCs are trained to monitor the implementation of the program,
provide case management support for DEI participants, & engage in
systems change activities throughout the grant period
SDSs are selected
Active outreach to WDAs begins
Information on the program is distributed to stakeholders & JSWDs
Coordination & integration of services
Participant customer choice
(2) Implementation
Grantees engage in a strategic process to define goals & objectives
SDSs are implemented with fidelity
DEI requirements are implemented with fidelity
DEI participants receive training, counseling, &/or job placement
support
Capacity to achieve positive employment outcomes
Employer engagement
Employer support & partnerships
Dissemination of effective practices & outreach to the disability
community
(3) Operational
DRCs collaborate with WDA staff to implement all program
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
35
requirements & have a clearly defined agenda that engages the
employer community, JSWDs, & WDA personnel & stakeholders
WDA recruits SSA beneficiaries through TTW
WDA TTW milestone payments are received
Implementation fidelity data is used to determine why program
outcomes are being/not being met
Coordination & integration of services, customer choice, &
employer support
Dissemination of effective practices & outreach to the disability
communities
Employer & JSWD outreach
Development of new or enhancement of existing practices
(4) Sustainable
WDA personnel & local area agencies & support services have
developed partnerships & collaborations that improve access to
employment & training services for JSWDs
WDA personnel have created impactful relationships that have
increased access to key supports services & employment
WDA has a realistic sustainability plan in place
WDA has resolved challenges that hinder progress to implementation &
sustainability
Promising practices are sustained after the grant period
WDA outcome payments are received
Employer partnerships & development of new or enhanced
strategies
Employer outreach to JSWDs
SDSs continue after the grant period
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
36
VI. Grant Implementation Round 5
A. State: California; Focus Area: Adults with Disabilities
Stated Goals and Objectives i.
The state’s Workforce Services Branch in the Labor and Workforce Development
Agency of the Employment Development Department administered the Round 5 DEI grant to
achieve systemic change and expand the capacity of WDAs to serve customers with disabilities.
This included significantly increasing training opportunities for JSWDs as well as for staff and
employers and continuing efforts toward serving adults with disabilities with universal AJC
accessibility across multiple dimensions. California planned to achieve the following objectives:
Expand the capacity of AJCs to use core and WIOA services as a part of IRTs that serve
people with disabilities;
Continue to develop a system of state-centric T/TA to build capacity and expand the use
of DEI service delivery strategies;
Increase WDA participation in TTW and Partnership Plus;
Provide access to Career Pathways for individuals with disabilities; and
Continue to demonstrate that AJCs can partially fund disability programs if they
complete the suitability determination application and become an EN.
California Round 5 included three WDAs that demonstrated a high capacity to serve
JSWDs due in part to their experience with the Disability Program Navigator project prior to the
DEI and the continuity of staff at the state and WDA levels. Prior to joining the DEI, DRCs often
previously worked in related occupations, including job coaching, mental health counseling, and
other occupations supporting individuals with disabilities. Some DRCs also had specific areas of
expertise, including working with veterans and ex-offenders with disabilities.
Each WDA designated one or more DRCs to provide direct services and case
management. California reported that they saw a lot of back injuries and carpal tunnel syndrome
among JSWDs as well as a fair amount of individuals on the autism spectrum. Other JSWDs
had problems with workplace appropriateness and mental health issues.
An important issue for the grantee was self-disclosure, as DRCs were required to
explain to JSWDs that disclosing a disability is about finding opportunities and
accommodations” and not about restricting access to services for individuals. DRCs created a
worksheet to help JSWDs make decisions about whether to disclose a disability or not.
State leadership created a position that served under the DEI State Lead that we referred
to as a “State-Level Supervisory DRC.”
26
This individual had responsibilities beyond the
administrative aspects of the DEI State Lead role and engaged in providing WDAs with support
26
Social Dynamics created the term “State-Level Supervisory DRC” to distinguish between the DEI State Lead and
the former’s subordinate leadership position.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
37
for program implementation, strategic planning, and training of DRCs on a quarterly basis. Two
further key components of California’s Round 2 grant to support DEI staff that continued into
Round 5 were the peer advisory team and the Traveling DRC. The peer advisory team planned
and conducted mentoring meetingswith DRCs and partners to discuss the implementation of
SDSs and other DEI requirements. The Traveling DRC provided oversight for each treatment
WDA, offeringinteractive trainings and practical information based on successful models
currently operating statewide.” One large WDA grantee created two DRC positions, one for
direct service and one to coordinate and build capacity. In the smaller WDAs, a single DRC
performed both of these roles.
DEI Service Delivery Strategies ii.
IRTs and Asset Development were the two SDSs of emphasis for California. IRTs
strengthened partnerships and leveraged resources among a variety of partners. Asset
Development focused on financial literacy combined with benefits counseling that facilitated
JSWDs to assess the impact of full-time work on their benefits. California also provided a
Workforce Development Manager who oversaw Community Services Block Grants and helped
strengthen integrated services for an array of practical challenges to employment such as eviction
notices, transportation, housing, shelter, and child care services. JSWDs were also able to access
emergency services, OJT and opportunities, AARP employment and related services, and Back
to Work 50+. The latter links individuals over 50 with information, support, training, and
employer access needed to regain employment and advance in an occupation.
DEI Round 5 led to an increase in the use of IRTs; all JSWDS participating in the DEI
also participated in an IRT. IRTs were implemented in a number of different ways, from group
meetings to telephone calls with WIOA-mandated partners. All JSWDs were co-enrolled in
WIOA and VR, and blending and braiding were also common practices, as was providing
resources for access to transportation, child care, and housing. DRCs also provided training in
résumé writing and job interviewing and information about TTW. Some JSWDs also received
job coaching and case management services both before and after employment was secured.
TTW was a priority for California and reportedly implemented with much success in
DEI, especially in the two largest WDAs. DRCs focused on workforce development issues and
collaborated with the Department of Rehabilitation (DOR) to support Partnership Plus and access
to training and employment using TTW. The strengthened relationship between the AJCs and
DOR helped the DEI gain access to SSA beneficiaries and collaborate to serve JSWDs enrolled
in TTW. Through a partnership with DOR, a “menu” was made available so that each JSWD
could select SDSs that would be most advantageous to them. The collocation of DOR and DEI in
most instances within AJCs and the establishment of all three DEI WDAs as ENs strengthened
the project’s support for TTW. Nonetheless, state agency personnel described challenges in
working with SSA that may have affected enrollment, utilization of workforce incentives, and
the fidelity of their implementation, including difficulties in accessing information and support.
Another challenge was the reluctance of some JSWDs to seek full-time employment due
to fears that they would lose their benefits, including SSI/SSDI, health, Medicare, Medicaid, and
Temporary Assistance for Needy Families (TANF). To address trepidations about working,
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
38
JSWDs were provided pre-employment workshops. These were reported to get JSWDs into job
mode: getting up, coming to the AJC, and getting their brains running again.” They were also
used as a tool to figure out whether JSWDs were ready for work: “If JSWDs cannot come every
day for a workshop or they find that overwhelming, they need to talk about why it was
overwhelming.”
There are not that many differences between SSA beneficiaries and JSWDs who are not eligible
for TTW. It is always about being out of work for a long time and wanting to get back to work.”
DRC
DRCs either became CWICs or consulted with CWICs to provide information on
Benefits Planning to clarify how employment income may affect a jobseeker’s benefits.
California instituted an online Work Incentive Coordinator certification, Disability 101, that
covered the basics of how returning to work would affect benefits. JSWDs were also encouraged
to seek full-time employment that included benefits. WDAs used Ticket revenue to fund staff
training, staff wages, and job coaches. Job coaches were available to individuals who needed
short-term, emergency resources to earn a certificate that could lead to training and long-term
employment. AJCs also prepared participant maps that focused on goal development and access
to technology to help individuals who would ordinarily “walk away feeling embarrassed or
illiterate.”
Individuals who have been out of work for a long time are hesitant about losing benefits. There
is a long list to get child care benefits and there is fear that they will go back to work and lose
their benefits before being able to afford to be independent. There was a lot of concern from
customers about losing health benefits. In their mind, they are at risk of losing them. They know
their overall existence is dependent on benefits. Therefore, the WDAs encourage customers to
look at full-time jobs with benefits. They may need to start in an entry-level position, but we want
to get them comfortable working full-time.” — DRC
WDAs also engaged in the identification of job candidates, development of résumés,
matching jobseeker skills to employer needs, and practice interviewing. Helping to strengthen
JSWDs’ support systems was also an important part of California’s Round 5 grant. JSWDs were
encouraged to engage with family, friends, and professionals who might support their job search
activities and training and employment aspirations. JSWDs were encouraged to communicate
with individuals who were already employed to help them access community resources. One
DRC mentioned that it takes a village to ensure that JSWDs have the resources, skills, and
motivation to become employed. This approach was integrated into their CP services (see more
below in the section on Career Pathways).
Implementation Summary iii.
The California DEI sponsored and organized quarterly meetings for T/TA, capacity-
building, and mentoring of Employment Specialists. The grant was overseen by the DEI State
Lead and DRCs. The quarterly meetings included WIOA partners, subject matter experts, DEI
staff, and WDAs that were prior DEI grantees. Topics included DEI SDSs, Partnerships and
Collaborations, organizational development, monitoring TTW implementation, and general
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
39
upgrading of the skills and knowledge of DRCs and Employment Specialists. The DEI State
Lead also coordinated regional meetings for skills development through group trainings. The
DRCs were full-time on DEI and spent their time on both building capacity of AJC Employment
Specialists to serve JSWDs and providing case management and direct services to jobseekers. In
the largest WDA, there were two DRCs, one for capacity-building and one for direct services.
To enhance their knowledge, DRCs received T/TA from the National Disability Institute
(NDI) and the Traveling DRCs. Trainings targeted areas of local need and included topics such
as DEI program implementation, disability etiquette, and JSWD counseling for Employment
Specialists. DOR provided a staff member who traveled to each WDA to provide support to
DOR customers. Other trainings were provided by Americans with Disabilities Act (ADA) staff
on a yearly basis covering employee rights, self-disclosure, hiring individuals with disabilities,
access to job accommodations, and enrollment in SSA and related programs.
JSWDs felt that there should be increased access to the Limited Examination and
Appointment Program for individuals who are members of AARP, as there was reportedly a
significant amount of discrimination against people who have disabilities.” The program
facilitates the recruitment and hiring of individuals with disabilities. It also provides alternative
ways to demonstrate qualifications for employment rather than through aptitude tests. A
workforce development manager for the Limited Examination and Appointment Program
oversaw Community Services Block Grants that integrated services into the AJC to deal with
challenging situations such as eviction notices, transportation to a shelter, and access to child
care.
During the implementation of Round 5, each participating WDA was enhanced with
upgraded physical and programmatic accessibility such as automated doors, access to
transportation, disability-friendly cubicles, parking locations, and AT. These enhancements in
the accessibility of AJCs and access to external resources were designed to increase participation
by JSWDs in WIOA Title 1 programs during Round 5.
California’s Round 5 DEI focused on multidimensional, universal AJC accessibility to
serve JSWDs with a collaborative approach that included the state’s Workforce Services Branch
in the Employment Development Department. To achieve systemic change and increases in the
availability of training for in-demand occupations, California expanded the capacity of WDAs to
provide JSWDs opportunities for education, access to services, soft skills training, CP
instruction, and collaborations and partnerships with employers and the WDAs. California also
developed T/TA for DRCs and Employment Specialists and expanded the use of DEI SDSs such
as IRTs and Asset Development, TTW, Partnership Plus, T/TA and skill-based interest
assessments, and documentation of employment experience and eligibility for SSA, Medicare,
Medicaid, and TANF support services.
Partnerships and Collaborations iv.
Integration and collaboration across WDAs implementing the DEI included state-level
interagency cooperation, particularly between the Labor and Workforce Development Agency
and DOR. These partnerships enabled consistent and productive messaging, resource leveraging,
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
40
and access to SSA beneficiaries and non-beneficiaries. California also created new partnerships
with local agencies. Agencies partnered with DEI by participating in T/TA, statewide
conferences, and webinars and assisting in career development and job placements for JSWDs.
Other collaborations included those developed with the Department of Developmental
Services and TANF, California Committee on Employment of People with Disabilities,
Veterans’ Employment and Training Service, California Health Incentives Improvement Project,
and community-based organizations and nonprofit service providers. Across the state, more than
two dozen agencies were identified as partners of the DEI.
The DEI State Lead partnered with DOR to generate informational material encouraging
businesses in developing alternative methods for JSWDs to apply for jobs since the California
Labor and Workforce Development Agency had concerns about the use of online job application
processes for JSWDs. Career Pathways road maps developed in conjunction with employers
served to both visualize employment journeys and motivate JSWDs to access employment and
training opportunities. Continuous supports from DRCs, AJC colleagues, and job coaches, such
as motivational interviewing, were used to “encourage aspirational discovery, self-regulation,
attainable goal-setting, and persistence.” WDAs tended to be customer-centric, meaning that the
employment process was individualized and driven by each individual “that walks in the door.”
Participation in the IRT process also was a source of client empowerment and choice. In addition
to disability-friendly modifications to AJCs, community-based partners helped implement self-
paced and multi-modal assessments, including use of AT.
California sought to educate employers about reasonable accommodations and the use of
Customized Employment and training through various means. DEI worked with employers to
develop strategies for employer engagement, which included collaborating with economic
development agencies and local Chambers of Commerce and educating employers on topics such
as tax credits, accommodations, and events such as job fairs and employer forums to increase
employer awareness and employment of people with disabilities. California also engaged with
the employer community through OJT sponsored by the Labor and Workforce Development
Agency and supported by several employers that played a key role in the DEI Employer
Engagement Community of Practice and through establishing partnerships with WIOA-
mandated partners. Employer partnerships included entertainment companies that hired people
with disabilities.
Employer engagement and training was also provided through the state’s Medicaid
Infrastructure Grant at San Diego State University and online employer human resources
trainings. California has an “employer servicesdepartment that reached out to employers to
provide training in OJT, CP, CE, and/or Entrepreneurship (Self-Employment). Through the
DEA, California connected with senior living facilities to hire people with disabilities into staff
positions. The Northern California Business Advisory Council included service providers and
employers that hired people with disabilities and regularly advocated hiring JSWDs by
upgrading their qualifications to meet the needs of employers and overcoming the fear that many
JSWDs have about losing SSA benefits. They also provided access to accommodations to
address employer concerns about hiring JSWDs and potentially insufficient accommodations to
support long-term placements beyond temporary work experiences and OJT.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
41
Career Pathways v.
California was one of the early implementers of Career Pathways training; it has
extensive experience with providing similar services for more than 15 years. Their services
include “industry-linked programs” and supports for JSWDs to help them gain access to in-
demand occupations that could lead to economic self-sufficiency. California’s long-standing
effort to enroll individuals in CP training included both educational and employment-related
training in partnership with local businesses and across a wide range of industries. California CP
training was provided at no charge to DEI participants, as were work experiences, OJT, and
internships. However, JSWDs and DRCs commented that individuals often waived CP training
in order to more quickly enter employment. According to one DRC, JSWDs tended to achieve
high employment placement rates by quickly accessing a job, rather than taking the time needed
to go through vocational training and OJT, even though better training could mean that the job is
more likely to be sustained.
Training is a harder sell because people do not see the benefit to putting income off.” — DRC
Although JSWDs were interested in immediate employment with only limited training,
there was some basic education that was needed to get them on the path to employment. With
OJT, we have the placement immediately, the client is getting a check, and AJCs can fill a
training gap during that period. Employers are committed because it will support their
workforce development needs. Through OJT, DEI participants have a $2-per-hour greater wage
than clients in general employment.”
Outreach and Dissemination vi.
California conducted outreach, recruitment, and referrals for individuals with disabilities
through its partner agencies, including DOR, the California Department of Developmental
Services, veterans’ agencies, TANF, and a wide range of community-based agencies. California
also conducted outreach and recruitment in support of “targeted populations” such as individuals
with complex employment situations. The state created linkages within WDAs to interact with
local agencies that provided support services and coordinated outreach efforts, including referrals
to and from partner agencies, disseminating through radio and newspaper ads, sending “email
blasts,” posting on social media/websites, and presenting to local community-based agencies,
religious organizations, schools, and libraries. California increased the number of JSWDs
entering CP programs and implementing Individual Employment Plans, OJT, and CE
opportunities. Quarterly meetings were a critical component in training and disseminating
information that all current and former DEI grantees (and some other non-DEI WDAs) were
funded to attend by the DEA state program. Many agencies and service providers were also
invited to participate in these quarterly meetings. The Traveling DRC also played an important
role in disseminating effective practices across the state.
Promising Practices: “Pathways for Successvii.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
42
The Traveling DRC provided 1-day trainings for AJC Employment Specialists in
separate modules to bring best practices and perspectives discovered through California’s DEI
and Disability Employment Accelerator (DEA) to improve service delivery to individuals with
disabilities in each treatment WDA.
Traveling DRC trainings are intended for various levels of AJC staff, management, and
relevant collocated partners who have a stake in improving services to individuals with
disabilities in WIOA-funded Job Centers. We would love to see some front desk staff, case
managers, and anyone else who might interact with individuals with disabilities, but [we] are
interested also in having supervisors, managers, and the like who can put action to some of the
discussions that will occur and understand what partnership and collaboration looks like on
paper.” — Traveling DRC
Round 5 Traveling DRC modules included the following:
One System for All and All for One — Serving People with Disabilities — All of Us Can,
But Do We?
To Politeness and Beyond!Disability Awareness, Sensitivity and Etiquette
Here an Acronym, There an Acronym — What is the DEI, DEA and IRT, and what do
they have to do with PWD and WIOA?
We Need a Driver — The Disability Resource Coordinator (DRC) Defined
Workforce, WIOA and Compliance Oh my! — Coordinated and Complementary
Employment and Training Services
Creating an Environment of Trust and Building Rapport — Disclosure 101
If We Build It…They Will Come — Examining Customer Flow and Improving Service
Delivery
Creating an Organization Nest Egg — Become an Employment Network (EN) and Build
Discretionary Funds
Rationale for Its Implementation
The Traveling DRC, in combination with the peer advisory team and its extant partners,
addressed the need for consistent and relevant T/TA across a wide range of topics and across
partners that would continue to provide services to JSWDs after the grant period. This position
was advantageous to the adoption and implementation of promising practices for DRCs and
Employment Specialists to implement training with fidelity.
Why the Practice Could Be Considered Promising
The Traveling DRC provided T/TA for DRCs and Employment Specialists in an effort to
build staff expertise and capacity to implement disability and employment-focused services. It
also provided opportunities for engagement with other agencies, such as DOR, in the use of DEI
services. The Traveling DRC, when combined with a peer advisory team and its partners, had
opportunities to share information and collaborate statewide. In essence, the Traveling DRC
differed from a DRC who remained stationed at an AJC within a particular WDA in that it
allowed DRC services to be available across a number of WDAs.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
43
Challenges and Sustainability viii.
As the DEI State Lead noted, “transportation is a huge problem in such areas as
Sacramento.” Most WDAs and the populations they served increased markedly, yet there was
little change in the state’s transportation system. For JSWDs who have limited access to
transportation, it may be difficult to travel to work that is outside of their metropolitan area as
light rail has not been expanded to some WDAs and employer locations.
A lot of thought about the job goes into how to get there. Employers ask for a driver’s license,
even if the job does not require driving. Many people with disabilities do not have a driver’s
license, which [requiring a license] performs a de facto screening-out function.” — DRC
Challenges to the implementation and sustainability of the DEI are limited due to the
state’s access to additional resources that fund DEA through state resources. DEA focuses on
partnerships between the AJCs and employers that, per the DEI State Lead, accelerate
employment and reemployment strategies for individuals with disabilities.” The DEA is similar
to the DEI as it focuses on connections with employers and engaging them in the development of
job training and employment opportunities.
The DEI team believed that the conclusion of DEI funding would lead to a significant
funding void and thus would mean limited resources for assessments, training, supports, and
dissemination. Nevertheless, there were many signs that disability awareness and promising
practices would be sustained as DEI State Leads and DRCs were intentional about integrating
DEI practices and policies into WIOA processes to forge system change. A major asset is the
DEA program, which funds all current and past DEI grantees to attend the state quarterly
meeting for peer learning, sharing effective practices, staff mentoring, capacity-building, and
sustainability.
The fruitful collaboration spurred by DEI, particularly with the California Labor and
Workforce Development Agency, appears to be continuing. Consequently, there will be “no
right door or wrong doorfor JSWDs to enroll in WIOA services. The use of IRTs should also
endure due to state leadership’s involvement in “cementing integration of this service delivery
strategy.” DRCs will continue at their respective AJCs at DEI’s end through DEA. In addition to
state funds to sustain DRCs and DEI practices, stakeholders mentioned tapping into Title I,
which provides WIOA services, and Title III, which authorizes Employment Service. TANF and
Supplemental Nutrition Assistance Program (SNAP) allocations are also being used by
California to provide access to employment and training of JSWDs. A prominent partner also
suggested that statewide philanthropy can help sustain Career Pathways programming. By
having DEA and TTW, DEI respondents said that DRCs have spread the roles out so our
services are easier to sustain over time because there is overlap with other programs and
because we have DEA and Ticket income, our services should be easy to sustain.”
Case management services also should continue as a key function of the grant as many
individuals benefitted from collaborations spurred by DRCs and WDA leaders. Many California
DRCs also became CWICs and were trained by ADA staff on employee rights, self-disclosure,
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
44
hiring, job accommodations, and enrollment in SSA and related programs. During the grant
period, WDAs also upgraded their physical and programmatic accessibility while state
leadership, in collaboration with the DEI State Lead and State-Level Supervisory DRC,
monitored access to WIOA services and programs. Given access to state resources, including the
DEA program, the state’s long-time experience in and commitment to supporting individuals
with disabilities, and the state’s focus on workforce development, it is likely that DEI/DEA
services will continue to serve JSWDs. California’s DEI addressed issues across micro
(individual), meso (services), and macro systems change concurrently.
B. State: Illinois; Focus Area: Adults with Disabilities
Stated Goals and Objectives i.
The Illinois Round 5 DEI proposed to achieve the following:
Create systems change within existing Career Pathways systems to increase participation
of individuals with disabilities in the information technology (IT) sector;
Increase awareness and involvement of JSWDs in CP and related employment and
training programs;
Achieve the following individual outcomes: post-placement employment through CP
completion, benefits counseling, job coaching, AT, and support services (e.g.,
transportation, child care, housing, etc.);
Allow DRCs to function primarily as case managers, mentors, and program navigators
who provide both employment and “life” coaching;
Ensure options for access to information technology CP and related fields as well as
training in life skills such as budgeting and soft skills; and
Allow DEI State Leads and DRCs to enroll JSWDs in information technology Career
Pathways programs through the DEI and extant training programs.
DEI Service Delivery Strategies ii.
Illinois DRCs developed IRTs and engaged in Blending and Braiding Resources to
provide SSA beneficiaries and other individuals with disabilities access to WIOA and/or SSA
services. IRTs were used to ensure that high school students enrolled in postsecondary education
and/or employment and CP and to facilitate opportunities for social support through bridge
programs that aided families in need of supportive services for their children with disabilities.
For example, helping a family access medical, educational, social, or other services could assist a
youth in achieving their employment goals, and coordinating multiple community-based services
could address the needs of an entire family and help them better support their child’s
employment plans.
DRCs also facilitated the transition process from school to training and eventual
employment in addition to providing outreach and marketing opportunities and ensuring students
had opportunities to participate in Career Pathways and receive appropriate supports and
services. Some DRCs engaged in ARC, which sometimes led to an IRT. ARC engaged
jobseekers early in the intake process, which included an application and documentation of a
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
45
jobseeker’s age and employment and training history, to discuss their key interests and concerns.
It was also used to identify specific activities or opportunities that may assist individuals with
disabilities to deal with challenges such as employment, socialization, and Asset Development.
Then, an assessment process was used to determine each individual’s skills and goals and if there
was a match between training, an internship, or OJT that may lead to employment. When
implementing ARC, DRCs used “scripts for each youth to target specific activities designed to
assist individuals with multiple challenges to employment by increasing access to services and
supports, including DEI SDSs. While some DRCs were less familiar with benefits counseling
services, the grantee did have access to these services through DRCs who completed their CWIC
certification.
Illinois DRCs coordinated the employment and training process for each JSWD and were
the contact persons who provided case management support and access to SDSs and IRTs. IRTs
were beneficial to JSWDs because they brought together WIOA-mandated partners as well as
other DEI SDSs, including financial literacy training, WBL Opportunities, CE, Asset
Development training, and social support through both DRCs and Employment Specialists.
Other SDSs included Blending and Braiding Resources, Partnerships and Collaborations,
flexible opportunities for T/TA, and systems change. The Computing Technology Industry
Association was a key partner and “leader in the national IT industry for providers of industry
certifications. It provided employment opportunities for individuals with and without disabilities
and offered a set of recently expanded Career Pathways. Youth with disabilities 18 years of age
and older were enrolled in SSI through the DEI. DRCs let them know early in the engagement
process about their eligibility and how working could be possible without affecting their
beneficiary status. During Round 1, DRCs had very limited understanding of TTW. Illinois
Round 5 was prepared for TTW implementation, although most youth were not old enough to
enroll in the program.
Participantsassessment results may assist in determining the relevance of certain
activities to employment, such as socialization with peers or opportunities to meet with
employers, résumé development, mock interviews with employers, WBL, Job Shadowing, CE,
and OJT. WBL and OJT were used to acclimatize jobseekers to the work environment and
provide “real-worldwage-earning employment. For youth with disabilities, the DEI
implemented Guideposts for Success together with WBL.
Implementation Summary iii.
The key components of the Illinois youth project included job training opportunities and
job placement support through “sectorial partnerships
27
in support of CP-oriented employment.
Sectoral partnerships were comprised of the DEI and AJCs, local employers, and the state’s
education community, including high schools, community colleges, and 4-year colleges. Illinois
27
National Skills Coalition. (n.d.). Sector partnerships. Retrieved from https://www.nationalskillscoalition.org/state-
policy/sector-partnerships
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
46
emphasized technology-related Career Pathways designed to help youth with disabilities obtain
in-demand skills and credentials designed to help them obtain a living wageemployment.
28
While DRCs focused on systems change and universally accessible AJC services, the
state implemented professional development plans for each agency partner, including WDAs and
the Division of Rehabilitation Services, and improved IT focused on “self-paced, accessible
online learning and classroom-based opportunities.Illinois also developed an employer
outreach strategy that included regional partnerships designed to facilitate enrollment in IT-based
occupations from high school through college. DEI SDSs included Individual Learning Plans
(ILPs), Individual Education Plans, IT-related WBL Opportunities, Job Shadowing, and CE.
These were augmented by ARC, OJT, internships, and CP tailored to youth with disabilities.
Other SDSs include Asset Development training, IRTs, Blending and Braiding Resources,
Benefits Planning, and Partnerships and Collaborations with WIOA-mandated and non-mandated
partners.
Partnerships and Collaborations iv.
Illinois Career Pathways included partnerships with the Illinois Pathway Initiative
Council, the Division of Rehabilitation Services, and local and regional partners, including the
Computing Technology Industry Association, local high schools, community colleges, nonprofit
agencies, and participating WDAs. The Department of Commerce and Economic
Opportunity also worked with the Division of Rehabilitation Services on “Have Dreams,” which
was the Illinois Task Force on Employment and Economic Opportunity for Persons with
Disablities. The xpanding information technology Career Pathways project was another
important component of the project The latter had representation from several state agencies
that coordinated statewide services for individuals with disabilities that included “employer-
driven pathways” in DuPage, suburban Cook County, and two Project SEARCH Transition-to-
Work Programs. The latter were led by local employers that provided a workplace that included
classroom instruction on soft skills and technical skills, career exploration, and worksite training.
DRCs appeared to be involved as partners with the high school transition staff “so by the
time they graduate, they’ll be comfortable. There’s a seamless transition from school to training
or whatever. We only have relationships with some of the high schools here, but the county is
big, so there are a lot of potential clients. We also focus on the community colleges because they
have a lot of JSWDs, so we go and see if we can help them out too.”
DEI employment outcomes focused on job retention and wage outcomes by providing
follow-along job counseling and support by engaging partner organizations. JSWDs were
connected to VR services, TTW services, and DEI supports such as case management, AT, and
transportation and child care assistance as needed. DRCs reported that they functioned primarily
as case managers, mentors, and program navigators who provided both employment and “life
skills.”
28
Partnership for Working Families. (n.d.). Policy & tools: Living wage. Retrieved from
https://www.forworkingfamilies.org/resources/policy-tools-living-wage
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
47
Illinois implemented TTW and was certified by SSA as an EN after completing its Round
1 grant. As with most early grantees, Illinois Round 1 struggled with EN implementation. By
Round 5, their EN was operational and provided opportunities for DRCs to enroll individuals in
Benefits Planning workshops; several became CWICs who supported the implementation of
TTW and collected milestone and outcome payments in Round 5. Illinois also worked closely
with AJC staff and WIOA-mandated partners and CP program administrators.
Employer Partnership v.
DEI youth with disabilities shared that they appreciated help with training and
employment opportunities. Youth were eager to meet and communicate about work and life with
employers, DRCs, and other people who were interested in their situations. Several mentioned
that developing a network of “friendswas an important part of employment as it provided an
outlet for socialization, building a reputation, and sharing information with colleagues. As one
youth beneficiary said,I like helping people, anything involved with getting customers napkins,
sauces, etc. What I don’t like about my job is that there’s not enough good communication
between staff and managers. They’ll come in and work a shift and ask us to stay longer so we
end up doing a double shift. And sometimes you tell one manager one thing, but they don’t
communicate with each other. So sometimes you have managers all scheduling you at different
times. But that’s what it is. It’s sometimes a challenge to do everything.”
I was having trouble rounding-up the shopping carts and they noticed I was having trouble, so
they worked with me. The movie theatre; I wouldn’t recommend them. I ended up losing that job
because I got sick and they weren’t willing to be flexible because I was sick during the first 90
days, which is a rule they never told me about. The training was very sparse; they never
explained how clocking in and clocking out worked, and I once had to leave work in an
ambulance, which I think made them nervous.” — Youth Beneficiary
I worked at an ice cream shop. It was a good experience, but the pay was below minimum
wage. I had to leave to get a different job at White Castle, which wasn’t a good experience. They
would schedule us and then send us home regularly. But I worked until I graduated high school,
then left and now I work at McDonald’s, which is OK.” Youth Beneficiary
Youth received VR services for accommodations (e.g., extended time on high school
tests and exams) as well as WIOA and Career Pathways training through community-based
agencies located in each WDA; several Illinois youth were noted to have received extended time
on reading and mathematics, automotive training and repair, and interface with local employers
to provide opportunities for Apprenticeships and Work-Based Learning Opportunities. Illinois
youth were dually enrolled in employer-sponsored vocational and academic training, which
provided individuals with a new perspective.
I can get help and be supported through all of these programs. I’m enrolled and pursuing a
career in early childhood education. But, there’re a lot of different routes. I like that it helps you
get skills that you need. I also like how there’s multiple paths to the program, like there’s
training and education programs. Some of us are in a program doing IT.” — Youth Beneficiary
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
48
Youth also had access to funds through WIOA that supported training needs. As one
youth noted, “the DEI helped me pay for my training. It gave me a new perspective, that I can
get help and support through all of these programs.” Many students pursued certifications in
early childhood education, computer and information systems, software development,
automotive repair, positions as teaching assistants at child care facilities, and supports such as
training in public speaking and community activism.
I started out in the 3-week workshop. I was originally going to take an internship, but I had an
issue. Not enough income. So, I changed at the last minute and am now going to college. I think
what the DEI is doing is great because it helps JSWDs get a job that pays well and helps me
understand finances. Going in to the workshops to get ready for college was great. I also liked
that they showed a variety of career options that didn’t involve academics.” — Youth
Beneficiary
I had a job before this and another one before that, but I left them both. I worked at a movie
theater making sure everything was clean, helping people get to the theater. I also worked at a
union job at a grocery store, and I bagged groceries, but then I switched departments because of
my health issues. My latest job was to make sure everything on the shelves looked nice and was
in the right place. They were really accommodating with my disability too. I had to be in and out
of the hospital, and I appreciate that they worked with me so I could do that.” — Youth
Beneficiary
The grantee’s employer outreach efforts included representation from several state
agencies that coordinated statewide services for individuals with disabilities. Additionally,
DuPage and suburban Cook County engaged school systems and community colleges with
Have Dreams,” a project that provided services to individuals with autism. The project
maintained relationships with employers, the Chamber of Commerce, and economic
development agencies to provide work opportunities for autistic youth and adults. As a
coordinated network, Have Dreams also partnered with local AJCs to identify and recruit
employers that could provide mentoring support services, including WBL Opportunities. The
state’s WDAs also help to identify championemployers that served as mentors in identifying
employment supports for information technology Career Pathways partnering employers.”
The grantee also implemented “Illinois Pathways,” which were statewide public-private
learning exchanges designed to support local and regional partnerships in each WDA. Local
WDA partnership efforts included those with employers, targeted business sectors, unions,
higher education agencies, and community-based agencies. When taken together, employer-
focused outreach combined with individualized employment and training services for JSWDs
with partners like the Computing Technology Industry Association helped employers connect
with DEI participants who had acquired locally in-demand skills.
Career Pathways vi.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
49
Career Pathways included services for youth in transition designed to develop skills in
information and related sectors.
29
Available occupational sectors included health care,
manufacturing, agriculture, transportation, logistics, and IT. Through various public-private
partnerships, the Computing Technology Industry Association offered an IT skill development
and employment system that facilitated employment soon after the completion of job training.
The state’s WDAs maintained task forces designed to increase opportunities for individuals with
disabilities through “employer championsthat provided access to full-inclusion employment.”
Recently, the Illinois Department of Commerce and Economic Opportunity conducted a
statewide study to identify key practices among schools, colleges, universities, and workforce
partners in job-driven, full-inclusion strategies.” Illinois also expanded its partnerships to
include VR and the Department of Education with the expectation that individuals with
disabilities who enrolled in Career Pathways would be equipped with soft skills, including
reading, writing, workplace etiquette, and access to living wage employment.” There was also
an ITboot campthat helped jobseekers become familiar with basic computer use and
technology.
Outreach and Dissemination vii.
Youth with disabilities were referred to DEI from WIOA Youth services, teaching staff
from local high schools, community-based agencies, and DRCs. Targeted outreach to youth was
delivered by DRCs in schools where they conducted presentations about DEI and AJC services
and supports. Learning forums that included information about career opportunities such as
Apprenticeships, WBL Opportunities, and internships also helped eligible youth learn more
about DEI services. DRCs used separate scripts” when presenting to youth, covering topics
related to job prospects, access to support services, ARC, and IRTs. Outreach was also done to
engage youth in the process of selecting a college or technical pathway. For example, if a youth
was interested in IT, they could enroll in a technology-related boot camp to familiarize
themselves with the IT field prior to receiving formal training.
DRCs also engaged in outreach to youth with disabilities through presentations about
DEI services, including college enrollment and CP opportunities. In addition, DRCs did outreach
to enroll youth in programs that provided an array of support services, from opportunities for
socialization to physical or psychological support.
29
Illinois expanded the capacity of its workNet Centers (i.e., AJCs) to improve the employment outcomes of
individuals with disabilities age 1624. Many WIOA youth do not attend school regularly. They also may hold a
high school diploma and are low income and/or basic skills deficient. Youth may also be involved in the juvenile
or adult justice systems and categorized as homeless, runaway, eligible for foster care (Section 477) and/or in a
designated placement (e.g., foster care, home care, juvenile program placement, etc.), pregnant/parenting, having a
disability, and/or low income.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
50
I was referred to the DEI from WIOA Youth services. They told me about it and that they could
help me get accommodations and things I need. I like the DEI because it helps me with stuff that
I have a disability with. They’ve helped me get extra time on tests. And I’ve been taking classes
and doing training to get ready for work. I’m also taking automotive classes and reading and
math.” — Youth Beneficiary
Promising Practices: “Pathways for Successviii.
Illinois DRCs had a strong partnership with VR that included a dedicated VR staff
member who made referrals to Career Pathways services, enrolled individuals in WIOA services,
and provided oversight of individuals who enrolled in community college.
Rationale for Its Implementation
DRCs worked closely with VR to make referrals to employment and training
opportunities, including Career Pathways. In addition to the implementation of CP, Illinois
provided WBL Opportunities, including Apprenticeships and internships, that allowed youth to
learn about employment and what it entailed (e.g., receiving remuneration, working in a group
environment, adhering to the requirements of the leader or “boss”). The advent of CP allowed
JSWDs to prepare themselves for the work environment, sample different kinds of occupations,
and develop interests that could lead to long-term employment.
Why the Practice Could Be Considered Promising
DRCs organized job fairs in which JSWDs learned about new opportunities for
employment and WBL Opportunities, Apprenticeships, internships, and Benefits Planning, all of
which were key components of the DEI that lead to self-efficacy and transition to full- or part-
time employment. “The benefit is that youth get to go to school and get any assistance that they
are eligible for from DEI or SSA, which is an important step toward developing the confidence to
be self-sufficient and maintain employment and independence.”
Challenges and Sustainability ix.
Illinois leadership initially expressed concern about the structure of the DEI grant and its
implementation. The grantee’s original idea was to develop partnerships across the state’s WDAs
and provide leadership and support to AJCs and JSWDs. However, the grantor wanted the DRCs
to coordinate outreach in each WDA to inform the AJCs and JSWDs of the kinds of services the
DEI offered and to focus on recruitment of JSWDs and integrating the AJCs so that all services
were available to both individuals with and without disabilities. The goal was a seamless,
integrated system for serving all jobseekers.
The grant has great intentions, but all the things that require of it take away from
serving JSWDs. Like meeting with all of the different agencies for the same thing, needing to
report information in a very specific way, made it less productive than it could have been.”
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
51
Initially, DRCs appeared to have limited knowledge of TTW, EN implementation, milestones,
outcome payments, and SSA work incentives. Although DRCs initially reported a limited
understanding of TTW, beneficiary recruitment, and the overall structure of TTW, DRCs
operated with limited involvement and training from the WDAs. This situation was improved
with the addition of a DEI liaison who focused on TTW and EN and was the initial point of
contact for individuals with disabilities. Eventually, DEI leadership successfully implemented
TTW and recruited beneficiaries.
While TTW was seen as a way to sustain the DEI, many youth did not have an assigned
Ticket because they were under the age of 18. DRCs commented that they had limited familiarity
with TTW, with one saying that they “know it has a goal of employment rather than receiving
benefits. I don’t know how it factors in with us, but I think we may pursue it. But I don’t have
much knowledge of it.” DRCs connected with the NDI TA center to discuss TTW and providing
access to community-based service providers through a statewide EN, including milestone and
outcome payments. Illinois also provided TA regarding the implementation of their EN. The
relationships that were created with VR since the implementation of Round 5 grant will continue
to be available after the grant period, and local colleges, such as the College of DuPage, will
continue to provide academic supports. However, it does not appear that all DEI WDAs will
have access to DRCs to implement TTW.
We’d have to work at TTW to become more familiar with it and sustain DEI in Illinois. We’d
need to involve partners. It may get territorial because people don’t want to give up their
involvement in TTW. However, the relationships we’ve created through the DEI will still be
around because everyone’s happy to have all those resources. The relationship with the College
of DuPage will be there too because we have common goals. DRC
C. State: Kansas; Focus Area: Adults with Disabilities
Stated Goals and Objectives i.
Kansas Round 5 was designed to build upon the foundation of the state’s Round 1 DEI
grant by continuing efforts toward universal accessibility. A primary objective of Round 5 was to
increase the percentage of JSWDs served by the public workforce development system with
training services more typically available to other jobseekersbeyond WIOA. Improving
accessibility included an emphasis on cultural change within the workforce system, with an end
goal of augmenting access to Career Pathways and certification for JSWDs.
The Kansas Round 5 DEI proposed to achieve the following:
Improve the postsecondary education and training outcomes for adult customers with
disabilities by:
o Increasing the percentage of adult customers with disabilities who are referred to
postsecondary education and training for existing successful Career Pathways;
o Increasing the percentage of adult customers who receive community and technical
college credentials along successful CP; and
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
52
o Increasing the percentage of adult customers who receive paid work experience
(internships, job sampling, etc.).
Improve the employment outcomes of adult customers with disabilities who obtain and
retain employment through a continuum of services to support education, training, and
employment success by:
o Convening individuals representing project partners, including community and
technical college staff, VR staff, community-based organization staff, WIOA staff,
and individuals from industry, to promote collaboration among multiple service
providers toward relevant skill development of customers with disabilities;
o Developing leadership training to community staff and improving and enhancing
assessment, accommodation, and coaching of students with disabilities;
o Through DRCs, training WIOA staff to establish and support OJT and other paid
work-based training opportunities;
o Through the DEI Technical Assistance Coordinator, providing a written report to the
State Lead detailing a review of quarterly performance data, individual customer
records, and case management documentation to ensure customers are receiving
relevant, timely, and effective services;
o Providing real-time employment and training data on JSWDs; and
o Increasing the percentage of KANSASWORKS customers who have disabilities who
are placed in OJT.
DEI Service Delivery Strategies ii.
Active Resource Coordination — or finding the best resources to meet individuals’
needs” — occurred regularly across treatment WDAs. ARC was the first step in the Integrated
Resource Team. ARCs identified targeted activities/programs designed to assist customers with
multiple challenges to employment by increasing access to services. A participant’s assessment
results assisted in determining such targeted activities/programs.
DRCs frequently engaged staff from other service delivery partners such as VR and
community-based agencies because we want everyone to get all the resources we have.”
Without DEI, ascertaining this information and additional resources would likely be “pushed
back to the participantwithout support from DEI and WDA personnel. One WDA partner
indicated that more regular meetings” could have improved ARC by providing opportunities for
JSWDs to select particular programs, activities, and Career Pathways to support their specific
employment needs.
IRTs were used frequently across the state. They included representatives from VR and
the state’s mental health system, where appropriate. Kansas reported that 160 IRTs were formed
during the grant period. However, IRTs were often conducted informally due to time and
resource constraints. In one WDA, a formalized meeting occurred twice because necessary
parties wereslammed” due to other commitments. However, when they occurred, those
meetings worked seamlessly and included DRCs, VR staff, WIOA case managers, job coaches,
and business services. While Kansas’s final quarterly report indicated IRTs were being initiated
by partnering organizations on behalf of shared customers, interviewed partners in one WDA
reported no involvement in IRTs.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
53
Blending and Braiding Resources happened consistently, including WIOA co-enrollment,
which helped fund certifications and support services. Combining WIOA, DEI, and VR funds
also allowed for extended training. One staff member asserted that blending WIOA and DEI
enabled youth programming to last 8 to 10 weeks to explore careers. Pell Grants also helped with
school-to-work transitions. As a DRC testified, funding cuts elevated the importance of Blending
and Braiding Resources.
While Kansas selected Self-Employment/Entrepreneurship, it was not a focus” of the
grant due to it being “difficult … to get up and running.” The AJC purportedly lacked business
development experience (e.g., about insurance), combined with high start-up costs, that would
make the implementation of Self-Employment a challenge for the grantee and JSWDs. AJCs did
refer clients interested in starting their own business to community colleges and organizations
that supported business development. One DRC “learned a little bit” about helping clients with
a Plan for Achieving Self-Support (PASS). DRCs also encouraged JSWDs seeking to explore
Self-Employment to consider “supplementary” income, such as from Etsy or farmers’ markets.
In terms of Asset Development, one AJC facilitated access to budgeting assistance, but it
did not offer any classes focused on Asset Development, nor did it proactively promote ABLE
accounts. A DRC asserted that the AJC prioritized immediate employment and training
opportunities rather than long-term asset building. Another WDA focused on Asset Development
that encompassed PASS plans and Individual Development Accounts. The third WDA did not do
any Asset Development with JSWDs according to a DRC. A related service, Benefits Planning,
was provided frequently to JSWDs.
There was evidence of TTW activity in treatment WDAs, as it was administered by the
state with some resources distributed to WDAs. One WDA was still researching the process of
becoming an EN, while another mainly referred TTW-eligible clients to available ENs. DRCs in
one WDA critiqued TTW for being “convoluted” in several ways: the ambiguous relationship
between employment and benefits and which clients were suitable; the delay and uncertainty in
returning revenue to local areas; and it being framed as a vehicle for immediate employment.
There was also uncertainty about AJC capacity to handle TTW in one WDA. Still, according to
its final quarterly report, Kansas exceeded its TTW expectations with 32 beneficiaries with
Ticket assignments and $79,307 in revenue.
The state’s Department of Commerce trained partners on CP programs. Significant funds
were directed to OJT training, internships, and apprenticeships. Project SEARCH was
implemented and DEI funds were made available for training from the Cerebral Palsy Research
Foundation and School of Adaptive Computer Training.
Implementation Summary iii.
Kansas’s Round 5 grant was administered by its Department of Commerce, which
oversaw both the workforce system and state economic development initiatives. State leadership
had significant and relevant experience, including with the Disability Program Navigator and a
DEI Round 1 grant. A unique feature of Kansas’s service infrastructure included managed care
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
54
organizations and three state agencies directly involved in Medicaid policy, service delivery, and
oversight. Round 5 included three treatment WDAs. One WDA had multiple concurrent DRCs
although DRC turnover was common across sites.
To accomplish its goals, Kansas proposed creating cross-agency partnerships through the
Kansas Department of Commerce, which was the linchpin of multi-agency work groups that
streamlined referrals, employer outreach, and data-sharing, including the Governor’s Technical
Education Authority, the Kansas Commission on Disability Concerns, and the Kansas
Employment First Oversight Commission.
Kansas engaged businesses in in-demand sectors, including engineering, manufacturing,
IT, health care, energy generation and distribution, and construction. Education and training
programs were provided, including Career Pathways with multiple levels of education so
students could make progress at their own speed. In addition, their education strategy included
flexible entry points and short-term credentialing that included remedial intervention and
responded to labor market information where employment opportunities existed.
Kansas measured systems change and performance with postsecondary education
partners to report DEI participants’ progress, including grades, attendance, and accommodations.
The Department of Commerce promoted CP programs, including through Family Employment
Awareness Training. DRCs facilitated IRTs, including staff from managed care organizations,
colleges, parole offices, and community-based organizations, and they encouraged staff to
engage JSWDs with high-quality intensive, supportive, and wraparound services, including
Benefits Planning, financial literacy, and career coaching along with universal accessibility
through alternative assessments and trainings (e.g., hybrid learning and cohort model), ILPs, and
service coordination.
Kansas implemented most features of its grant application, though there was not much
evidence of a monthly reporting mechanism for IRTs or ILPs. A DRC asserted that her WDA
was the only areain Kansas to implement CE. An OJT opportunity went well and an employer
wanted to keep the client, but for another position. A new job description was created to “fit his
ability and needs.” However, a partner in this WDA desired more CE. Another WDA used group
Discovery as a tool for Career Pathways.
Partnerships and Collaborations iv.
Coupled with WIOA, DEI strengthened the state workforce development system’s
collaboration with VR and fostered many new partnerships throughout treatment WDAs. As
well, DRCs served on Commerce steering committees in their respective service areas. However,
in one WDA, there was some “pushback” from providers leery that DEI was encroaching on
their funding.
Heightened collaboration fomented productive cross-trainingespecially between the
core partners, KANSASWORKS and VR — which enlightened staff about additional funding
sources and services for JSWDs. Cross-training was promoted by state policy frameworks,
including integrated service delivery and functional management, which included management
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
55
that maintained authority over organizational units in an agency. In the case of the DEI, it was an
approach to selecting WIOA and Workforce Innovation Fund services.
DEI also helped codify linkages with postsecondary and career and technical schools.
Schools became more aware of JSWDs and AJC services. WDAs also cultivated synergy with
Project SEARCH through DEI, enhancing information-sharing and assistance to mutual clients.
DEI strengthened bonds with Behavioral Health Services agencies, local nonprofits, and
managed care organizations, which led to more AJC programming, more efficient job placement,
and easier access to partners and supports for JSWDs. One DRC solidified bonds with reentry
programs.
IRTs and staff meetings increased partnerships and referrals. A WDA director
appreciated workforce representation at board meetings and praised core partner meetings for
facilitating “comprehensive” service delivery, “smooth” referrals, and enhanced relationships
with job developers and college counselors. Nonetheless, one AJC partner suggested there could
be more cooperation to increase mutual awareness.
DRCs were the linchpin of expanding partnerships. DRCs connected stakeholders with
the right person in the system” and elucidated “how to use different assets from different
agencies.” In one WDA, multiple interviewees credited the DRC role with improving
KANSASWORKS’s relationship with VR. DRCs also improved rapport with referral sources.
However, ongoing turnover in the DRC role frustrated relationships with some partners.
State policies and leadership encouraged collaboration. For instance, the DEI State Lead
met with VR leadership to promote referrals to WDAs. A DRC valued state leadership as a
resource” and for “connections.” Per the DEI State Lead, WDAs were much more involved due
to WIOA’s access to its services and supports. Co-enrollment — particularly for young adults
but also for VR and Title I clients — expedited integration of JSWDs, while state-based projects
to improve data-sharing and referrals were also underway toward the end of the grant period.
Cross-agency staff training maximized universal access, established eligibility, and
provided career services to JSWDs. Support services were provided by employers, community-
based agencies, and the DEI. Treatment WDAs made progress with the state in upgrading
physical and communications accessibility, including installation of automatic doors, refined
language for engaging JSWDs, and “plenty of resourcesfor AT. DEI also furnished resources
that provided clothing for employment opportunities, trainings, and services for a broader array
of employment pathways, including social work, child care, and taxidermy.
Treatment WDAs indicated that they needed more training and capacity-building in
benefits counseling. JSWDs generally had access to a benefits specialist, but AJCs discussed
staff becoming CWICs as well. In one WDA, DRCs expedited “constantstaff training,
including on co-enrollment. DRCs commented that still more training in disability etiquette and
eligibility would be beneficial across the state. One of our WDAs could use more regular
trainings on serving JSWDs.”
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
56
Two of three WDAs demonstrated customer-centric qualities, such as providing
information on job accommodations as JSWDs sought employment and encouraging them to
articulate their employment goals. In one WDA, a participant focus group yielded mixed
feedback about customer choice. Positive testimony included characterizations of staff as “very
friendly and supportive of individuals by securing internships in relevant fields. Jobseekers also
appreciated vastnetworking resources.” However, there were multiple critiques from JSWDs in
one WDA, including a lack of clarity about wages upon placement and long-term opportunities
and a “one-size-fits-all model” approach to job development, including mandatory completion of
unnecessary classes. By contrast, the DRC from one WDA described efforts to streamline client
assessments at intake to make the process more customer-directed. She also reflected on how
DEI funding allowed clients to explore more career options. Further, she encouraged enrolling
clients in as many programs as possible, “so long as it is appropriate for the customer.”
Though a DRC stated she was not even familiar” with universal design, treatment
WDAs exhibited several inclusive features. One AJC facilitated both in-person and online
coursework through partners, depending on a client’s preference. AT in treatment WDAs
included Microsoft Reader and other screen-reading applications, as well as adaptations like
putting programs on participants’ phones or adjusting computer layouts. Collaboration with local
community colleges expedited appropriate AT. Workshops were multi-modal, including
PowerPoint slides, written material, videos, and interactive soft skillsactivities.
Goal-setting was promoted through user-friendly job-seeking platforms like O*NET
OnLine and guidance on budgeting. Career exploration inventories and identification of
transferrable skills reflected efforts to promote autonomy and relevance in job searching. DRCs
also varied demands and resources based on a client’s background, interests, and skills.
DEI built upon a state initiative that incentivized employers hiring JSWDs. DRCs were
influential in rolling out the program. In one WDA, the AJC was proactive in engaging
employers, with one DRC seeing his primary role as “to reach out and educate employers.” This
AJC interacted with employers through a variety of vehicles, including direct outreach, planned
meetings, and job fairs. Business services also facilitated employer outreach, particularly for
young adults, and a partner assisted in this endeavor. A staff member affirmed that the DRC was
integral to understanding which employers were interested in hiring JSWDs. Outreach to
employers involved framing JSWDs as being “more dedicatedthan jobseekers without
disabilities.
Employers confirmed this AJC’s proactive engagement and depicted AJC outreach as a
catalyst for relationship-building with JSWDs. The AJC also helped employers understand
JSWD needs and skills and provided assistance with any issues. WDAs conducted outreach to
businesses; while one DRC discussed considering an employer as “a partner,” there was little
evidence of employer involvement in AJC strategic planning.
DRCs identified several challenges to employer engagement. Securing buy-in to hire
JSWDs, including through provision of OJT funds, was challenging due to uncertainty about
liability, the practicality and cost of accommodations, and disability etiquette. DRCs suggested
small businesses may be more receptive to outreach than larger corporations due to a greater
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
57
likelihood of engaging an individual comfortable with relating their own familiarity with
someone with a disability. They also indicated employers were more open to hiring younger
JSWDs. An employer identified the obstacle of needing specialized and technical skills from
JSWDs. Some employers were also reluctant to get involved with OJT opportunities. Employers
showed preference for paid work experiences or direct placement. Partners bemoaned confusion
about establishing OJTs despite a multitude of potential worksite connections and called for
additional information and training in this facet.
Career Pathways v.
DEI financing supported longer training programs than WIOA and was more conducive
to stackable credentials and career ladders. In one WDA, DRCs were conversant in career
assessment options, including WorkKeys, the O*NET Interest Profiler, and TABE for youth.
This AJC also harnessed labor market information and job databases to provide JSWDs with
pragmatic insight. According to the DEI State Lead, Round 5 was instrumental to formalizing
relationships with postsecondary institutions and to enhancing accommodation policies at these
schools. These improvements were crucial to supporting WBL and work experiences
implemented statewide, including for youth and epitomized by Project SEARCH internships.
DEI facilitated 42 Project SEARCH internships in Kansas.
According to Kansas’s final quarterly report, 152 DEI participants completed classroom
training leading to certification, and 138 individuals participated in paid work-based training.
About half acquired Microsoft Office Specialist certifications; customer service training was also
popular. Per Kansas’s DEI State Lead and DRCs, enrollees acquired and maintained employment
for at least 6 months at a higher rate than anticipated and at a higher average hourly wage than
projected.
Outreach and Dissemination vi.
DEI state leadership helped form the Transformers Coalition. The Coalition was designed
to improve access to and knowledge about transition services for youth with disabilities,
including through town halls across the state. Treatment WDAs engaged in multifaceted
outreach. For instance, DRCs and AJCs reached out to local nonprofits and relevant county
agencies to discuss available services and employment opportunities such as WBL and OJT, as
well as the potential for collaboration and resource leveraging to assist JSWDs. An “accessibility
committeewas another venue for outreach, and DRCs promoted DEI through other committees.
In one WDA, DEI was discussed at a board level” to reach employers. Events like job fairs for
JSWDs and the Midwest Ability Summit also enabled outreach. DRCs were proactive in
engaging school districts and county governments, distributing flyers with information about
DEI.
To broadcast best practices, the Department of Commerce produced video interviews
exhibiting the promise of OJT. Interviews included DEI participants and employers from Kansas.
The state’s CP efforts through DEI were also amplified by NDI in a webinar, highlighting
significant CP enrollment. Local dissemination revolved around case management for JSWDs
and encouraging staff to be “creativein serving this group. A partner cited the Project SEARCH
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
58
committee as another channel for information-sharing. Cross-training also allowed for
dissemination beyond AJCs. In addition to productive individual relationships with employers,
one AJC shared information with employers via business services and the Society for Human
Resource Management.
Promising Practices:Pathways for Successvii.
One DRC developed a “Pathways for Success” curriculum in partnership with mental
health providers. The curriculum emerged from IRTs and was targeted to jobseekers with
multiple barriers to employment, including mental health challenges. Pathways for Success
integrated elements of Discovery, CE, career exploration, résumé writing, soft skills
development, mock interviews, benefits counseling, and Asset Development. Participants met
biweekly with workforce and mental health staff. Several clients also developed PASS plans.
Rationale for Its Implementation
Pathways for Success precipitated other agencies and workforce regions to pursue this
model. An intellectual and developmental disabilities provider adopted the model and formed
two cohorts. Two DRCs also partnered to form a regional Pathways to Success group that
continued to meet monthly.
Why the Practice Could Be Considered Promising
The curriculum addressed a significant gap in services for JSWDs, particularly the many
who had mental health challenges and multiple barriers to employment. Without tackling each
obstacle with comprehensive services and supports, the employment journey would likely be
daunting.
Challenges and Sustainability viii.
Like many DEI grantees, the most trenchant challenge for Kansas was turnover in the
DRC role. This prevented optimal training and collaboration. Capacity gaps also remained in
benefits counseling, AT utilization, and serving JSWDs with multiple barriers. Capacity
constraints were also evident in one WDA that lacked resources to regularly assemble IRTs. In
addition, there was uncertainty about disability disclosure and serving individuals with severe or
multiple disabilities, though there was progress in deciphering specific disability information.
Multiple DRCs spoke about the challenge of navigating “helicopter” parents of youth with
disabilities to ensure youth voiced their input. Another staff member stated, Our challenge is to
understand their needs” and to “channel [programming] to their needs.” Meeting one-on-one
with clients helped in this regard. There was no evidence of JSWD involvement in strategic
planning for the design of products and services for JSWDs.
Several features of treatment WDAs bode well for sustaining DEI practices. Multiple
DRCs had significant case management backgrounds, so they understood well the delicate
balance between service provision and systems-level training and coordination. In one WDA,
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
59
DRCs “[drew] a line with case managementbut were “pulled back in several times” to provide
support and answer questions: there’s always that mixing and molding.” A State Lead professed
that Round 5 DRCs did “not allow themselves to become case managers. They really are
resources for the case managers and other folks in the system.” According to a quarterly report,
case managers became more aware of other agencies and independently connected with partners
to create IRTs for JSWDs. This was corroborated in a post-grant sustainability call.
One DRC described her role as coordinating case management between agencies. DRCs
also spent much time fostering partnerships and conducting outreach to agencies; some explicitly
articulated this as their primary function. The DRC role encouraged sustainability by facilitating
awareness, training, and relationships. Moreover, former DRCs were integrated into AJCs —
two were still called DRCs — and were accelerating adoption of IRTs. AJC and some DEI staff
were also retained. Nonetheless, much turnover in the DRC role undercut long-term embedding
of DEI practices, as confirmed by the DEI State Lead in a post-grant call.
In terms of financing to undergird functions of a DRC after DEI, there was mixed
feedback. A staff member in one WDA suggested it would be “very challenging” to sustain the
DRC role, a reality that would also hinder collaboration. Many strengthened partnerships could
endure, such as with VR, but others could “suffer” without “the glue” of a DRC. The absence of
a DRC would also jeopardize AJC proficiency in engaging JSWDs “in an appropriate way.” On
the other hand, a State Lead communicated plans to fund DRCs as an “Other Shared Cost”
through partners in accordance with WIOA regulations. Reentry funds and data showcasing a
DRC’s impact were other potential avenues for sponsoring DRCs.
Beyond the DRC position, an interviewee from one WDA predicted that outreach and
training focused on JSWDs would be sustained by KANSASWORKS. The DEI State Lead
offered that JSWD engagement with Workforce continued to expand. The state’s final quarterly
report suggested training on DEI practices would be afforded to WDAs that were not involved in
DEI. Cross-system collaboration also became a feature of Kansas’s workforce development
system. Moreover, IRTs were replicated statewide, including by the state’s Service Guidance and
Support Teams, due to training through DEI. Per the State Lead, IRTs were the most lasting
legacy of DEI. OJTs continued to be promoted through television commercials and social media.
Post-grant, Kansas hired a consultant group to explore customer-centered design with employers
and customers. A State Lead was also charged with creating a Business Leadership Network
(BLN) and connecting with employers.
Blending and Braiding Resources will continue to be crucial, especially for a treatment
WDA with fewer resources and to ensure JSWDs receive both workforce and VR services.
However, a DRC submitted that local momentum to become an EN will subside without DEI,
restricting TTW revenue. After the grant ended, a State Lead did suggest that Kansas saw a
significant increase in Ticket holders. There were multiple potential additional avenues for
financing DEI practices. A State Lead indicated Pre-Employment Transition Services funding
would persist through an agreement with VR. A DRC mentioned the possibility of tapping into
the End Dependence Kansas grant and relying more on partners. Moreover, an AJC staff member
cited a state grant to fund IRTs. A partner intimated that Project SEARCH financing was solvent
through the county government. Finally, to translate DEI’s local impact into an outcome-based
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
60
regional and statewide sustainability strategy, Kansas also scrutinized local DEI data to better
understand customer performance and service utilization relevant to all five WIOA titles.
30
Kansas stands out in its alignment of federal, state, and DEI policy frameworks. This
synergy fostered the collaboration inherent to DEI, as exemplified by fruitful partnering, IRTs,
and Blending and Braiding Resources. These processes are likely to sustain well beyond the
grant’s end, as they had spread statewide. DEI contributed to significant JSWD engagement with
KANSASWORKS and partners, though capacity, resource, and training gaps linger. Project
SEARCH and Behavioral Health Services were unique partners, and Kansas leveraged these
relationships to cultivate pathways for clients with substantial hindrances. Ample employer
engagement led to opportunities for Work-Based Learning, including OJT. Still, OJT presented
complications that should be resolved. There were also implementation issues with TTW, and
the relationship between state and local TTW administration should be addressed.
D. State: Massachusetts; Focus Area: Adults with Disabilities
Stated Goals and Objectives i.
Massachusetts’s Round 5 “Pathways to Employment” project proposed to serve 165 adult
JSWDs, with a focus on bettering job placement rates by increasing access to community college
and Career Pathways in manufacturing, health care, and hospitality. “Better services” through
service coordination and universal design were seen as pivotal to this goal. The state laid out
seven objectives for the project:
1. Stimulate cohesion and collaboration among providers and agencies serving JSWDs;
2. Expand access to technical training and education in targeted sectors;
3. Increase the amount and diversity of employers hiring JSWDs;
4. Raise awareness among employers about the benefits of hiring JSWDs and about
available support services;
5. Provide training and support to JSWDs in navigation of career development, education
and training, and disability service systems;
6. Augment access to short-term subsidized work through WBL, OJT, and other direct work
opportunities; and
7. Support job placement and retention in unsubsidized employment.
Five additional “systemic changes” were presented as crucial:
1. Raise awareness among providers about current CP offerings and how to integrate their
services with existing initiatives;
2. Create trainings for employers, CP instructors, and AJC staff on supporting JSWDs;
3. Forge inclusivity and integration in CP programs, including by reflecting input and needs
of JSWDs;
30
WIOA includes five titles: Workforce Development Activities (Title I), Adult Education and Literacy (Title II),
Amendments to the Wagner-Peyser Act (Title III), Amendments to the Rehabilitation Act of 1973 (Title IV), and
General Provisions (Title V).
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
61
4. Augment support and ENs for students who complete CP; and
5. Engage employers through a “continuous communication loop” with educators and
trainers, “inclusion planning,” and job development and coaching to align their needs
with training and job placement strategies. Build employer bonds and translate employer
information to lead to “on-target” training and a greater likelihood of “pre-employment
exposure,” OJT, and placement. Several strategies were proposed to engage employers:
a. Business leaders were to present quarterly about needed skills;
b. Information sessions for at least 30 business leaders in manufacturing, health care,
and hospitality;
c. A biannual course for employers on recruitment and hiring, job accommodations, and
creating an inclusive environment resulting in an Inclusion Plan;
d. Consulting on job carving and development, accommodations, assessments, and
internships; and
e. Training on “natural” supports to provide long-term job coaching; five co-workers to
be trained to become natural support job coaches to assist in job stabilization.
DEI Service Delivery Strategies ii.
Massachusetts planned to achieve its goals and objectives through several service
delivery approaches. To supervise implementation, each treatment WDA established a DEI
committee that included partners and employers, such as the University of Massachusetts at
Boston, Institute for Community Inclusion (ICI). In addition to setting project guidelines, the
DEI committee promoted resource blending and braiding and gathering partners to participate in
an annual conference. The Department of Career Services (DCS) and the Massachusetts
Rehabilitation Commission (MRC) co-enrolled JSWDs; three other agencies or grants were
tapped for resource leveraging (e.g., Trade Adjustment Assistance Community College and
Career Training, Department of Mental Health, and Veterans’ Employment and Training
Service).
The WDAs featured DRCs, career counselors, and business services representatives, and
they also included mental health providers in IRTs. A business services representative asserted
that IRTs were essential to optimizing knowledge and resources for JSWDs, including about
accommodations. In another treatment WDA, IRTs included MRC, the Massachusetts
Commission for the Blind (MCB), and career development coaches from local community
colleges and nonprofit providers. IRTs were implemented more formally in the third treatment
WDA, which included wraparound services and training and employment opportunities.
Blending and Braiding Resources included partnering with VR, particularly for training
and work experiences. In one treatment WDA, DEI funding enabled VR clients to complete
trainings, primarily in access and registration. This AJC often referred JSWDs to VR to facilitate
transportation assistance, including cab vouchers. Toward the end of the grant, this WDA
overcame limited training funds and provided the same level of services to JSWDs by leveraging
WIOA Adult funds.
Through alternative assessments in concert with career exploration and benefits
counseling, the state expected to achieve improved assessment, matching, and referrals to
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
62
appropriate programming with supports up-front.” Available assessment tools included
WorkKeys,
31
Transferable Occupation Relationship Quotient (TORQ)
32
software, and
Accuplacer, which is a school readiness program.
33
IRTs within each treatment WDA developed individualized service and career action
plans and integrated these plans within existing services (including TTW). Plans involved an
eco-map of existing systems and supports that outlined the role of each partner. In terms of
training, strategies featured contextualized learning, compressed training awarding credit for
prior learning, dual enrollment, hybrid learning approaches, and job carving.” Community
college students also joined “learning community clusters that offered small class sizes that
included team projects. DEI pathways included both certificates for entry-level employment and
stackable credentials.
Round 5 included a train-the-trainer approach to financial literacy through which all DEI
staff could promote Asset Development services designed to help an individual purchase a car or
home. Finally, Massachusetts hosted their second disability employment conference. DEI
stakeholders participated and employers advertised job openings. This conference helped with
replicating DEI practices throughout the state’s 16 WDAs.
Implementation Summary iii.
Massachusetts Round 5 built upon a CP infrastructure incubated by the community
college system in Round 3. Round 3 established the groundwork for an “expedited start-up
process” in Round 5. In addition, Round 5 benefited from statewide collaboration around CP that
germinated as a result of the state’s Transformation Agenda, a systems- and industry-based
approach supported by a $20 million USDOL grant. A state-level, cross-agency Coordinating
Committee oversaw the DEI grant. The committee was led by the Executive Office of Labor and
Workforce Development and the Executive Office of Health and Human Services and included
representatives from DCS, MRC, the Department of Mental Health, Veterans Services, the
Massachusetts Commission for the Deaf and Hard of Hearing (MCDHH), and Developmental
Services. This committee was a “permanent component for cross-agency coordination to support
the needs of JSWDs.” Round 5 also included regional working committees to identify
resources and services for JSWDs. Multiple AJC staff members, the DEI State Lead and DRCs,
and employer partners were involved.
Notwithstanding much administrative restructuring and staff turnover, treatment WDAs
showcased the necessary leadership and capacity to facilitate positive employment outcomes for
JSWDs. Accessibility was a major focus that coincided with the state’s AJCs’ undergoing
certification processes. Compliance with WIOA Section 188 guidelines drove much of
accessibility planning, particularly in one treatment WDA. Local and national TA furnished by
31
ACT, Inc. (n.d.). WorkKeys assessments. Retrieved from https://www.act.org/content/act/en/products-and-
services/workkeys-for-employers/assessments.html
32
O*NET Resource Center. (n.d.). Transferable Occupation Relationship Quotient TORQ ™ from Workforce
Associates, Inc. Retrieved from https://www.onetcenter.org/paw/entry/125
33
College Board. (n.d.). What is ACCUPLACER? Retrieved from https://accuplacer.collegeboard.org/
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
63
ICI and NDI was instrumental in guiding treatment sites to alignment with Section 188,
including through cross-training. Massachusetts’s Office on Disability also conducted an ADA
assessment and reviewed AT access at participating AJCs.
To actualize accessibility, one treatment WDA focused on training, procedures,
accountability, facility upgrades, and communications. It completed three accessibility
assessments, scrutinized the 188 Disability Reference Guide, and developed an action plan to
remediate accessibility issues. The plan included two full-day, all-staff trainings, which were
conducted with assistance from ICI and focused on best practices for universal design and
services. Attendees included MRC, MCB, MCDHH, and Work Without Limits (WWL).
Representatives from NDI and the LEAD Center served as trainers.
In addition to AJC signage changes to improve communications accessibility, one WDA
enhanced its MassHire website to make it more inclusive, though it had insufficient funding to
completely transform the site as desired. The AJC’s lobby and wheelchair ramp were redesigned
to better accommodate people with a mobility impairment. In another treatment WDA, the AJC
was relocated during the grant period and subsequently assessed for accessibility by an ADA
coordinator. The AJC also conducted Section 188 training and received accessibility guidance
from NDI.
Partnerships and Collaborations iv.
Round 5 built upon collaboration and integration spurred by earlier DEI grants and access
to WIOA services. In addition to serving as state partners, ICI and WWL nurtured local
connections. ICI was integral to incubating partnerships in treatment WDAs, and WWL
connected treatment sites to other disability initiatives in the state, including Partners for Youth
with Disabilities’ Campus Career Connect program and the National Organization on
Disability’s Campus to Careers pilot. Round 5 bolstered bonds with MRC, which is co-located in
treatment AJCs at least monthly, including through co-enrollment and by treatment sites
attending monthly MRC meetings to encourage referrals. Treatment WDAs also partnered with
MCB, MCDHH, the Department of Developmental Services, the Department of Transitional
Assistance, and the Massachusetts Department of Elementary and Secondary Education’s special
education unit.
Collaboration was encouraged by monthly meetings coordinated by the State Lead, which
were either in-person or by teleconference and attended by DRCs, WDA leadership, ICI, WWL,
and NDI. Successful placements and upcoming events were discussed, and participants also
resolved difficult cases and other implementation challenges. One DRC described the meetings
as great … I think you learn a lot from hearing what other sites do.” In between monthly
meetings, DEI leadership communicated regularly with DRCs and Employment Specialists
through T/TA workshops. Two treatment WDAs also partnered to facilitate internships.
At the local level, one treatment WDA established a Disability Employment Coalition
since there was no existing regional collaborative like in other treatment WDAs. The coalition
met regularly and grew throughout the grant, incorporating employers, schools, family assistance
providers, youth transition services, Adult Basic Education (ABE), Job Corps, Easter Seals, and
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
64
other social service agencies. The coalition helped match JSWDs with available trainings and
positions and expedited IRTs and referrals. In addition to the coalition, the DRC worked closely
with MRC, MCB, the Department of Developmental Services, and the local community college
and high schools to leverage resources, implement IRTs, co-enroll, facilitate trainings, and
recruit and place candidates.
In another treatment WDA, DEI contributed to “a much more collaborative approach
and “a whole lot of communication.” The AJCs and a local provider that specialized in training
jobseekers with significant disabilities partnered on designing and implementing trainings and a
referral poolof JSWDs and employers. For example, one AJC trained the provider’s youth
clients for retail jobs. AJCs also united with veterans’ representatives and a mental health service
provider. A DRC remarked that communicating with referring agencies was a challenge.
The third treatment WDA made deliberate efforts to ensure DEI was not a siloed
program.” Accordingly, about 90 to 95percent of DEI clients were co-enrolled in WIOA,
Title I, or other programs. As conveyed by career counselors, however, a caveat to this strategy
was the stringent employment outcomes required by WIOA. This WDA also fostered “a really
strong relationship with MRC, including through many referrals, alignment of trainings, and
constant communication.” The DRC also presented at MRC staff meetings. Each treatment
WDA also conducted cross-agency training, as well as training of provider staff that included
information about JSWDs, opportunities for soft skills training, and information about TTW.
Transportation was cited as a hugebarrier to physical accessibility. This dilemma was
mitigated by the local staffing agency instituting a new van route. Additionally, with the support
of DRCs, the WDA secured funding from a local bank to help with transportation. AJC staff also
attended a statewide transportation conference as part of ongoing efforts to tap into state
resources and identify transit options. In another treatment WDA, a DRC was compelled to refer
JSWDs to MRC because we cannot provide the transportation that someone might need,”
whereas MRC could supply cab vouchers. Braiding funds enabled MRC to finance transportation
and the AJC to fund training. A lack of transportation access was compounded by many
vulnerable jobseekers having to secure a driver’s license or a vehicle; they may be able to go to
work tomorrow if they had transportation.” The AJC also did not have accessible parking. To
address these issues, DRCs worked with WDA staff to engage MassRIDES and advocated via
the state’s transportation committee. One WDA offered travel training for JSWDs, which
provided road maps to frequently visited locations selected by JSWDs.
ICI and NDI helped with mental health and anxiety accommodations and clinical
supports to facilitate programmatic accessibility. ICI also evaluated the intake process at
treatment AJCs. Treatment WDAs made substantial efforts to serve all JSWDs, including those
with significant additional barriers (e.g., substance abuse and criminal history). One treatment
WDA featured a partner that specialized in serving JSWDs with intellectual and developmental
disabilities and who neededmore significant supports,” as well as serving referrals from MRC
for the hardest-to-place” individuals. A DRC from this partner indicated that AJCs struggled to
serve and place jobseekers with intellectual and developmental disabilities and that such
jobseekers were less likely to seek full-time employment and/or Career Pathways training due to
the skill and training requirements. The DRC argued AJCs were not designed, staffed, or funded
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
65
to provide the necessary “level of individualization or intensive one-on-one servicesto this
population. Still, DEI facilitated service coordination to engage AJC staff and JSWDs.
Another treatment WDA served clients with substantial barriers to employment,
including those who were deaf or hard of hearing, blind or visually impaired, and facing mental
health challenges. According to the DRC, nobody [at the AJC] knew about MCDHH, [MCB],
or working with blind and deaf individuals” prior to DEI. MCDHH supplied interpreters and
signers, and MCB collaborated with WWL and employers to implement accommodations such
as JAWS screen readers. Also, MCB committed to providing additional resources for AT to the
AJC while MCDHH raised awareness about AT for hard-of-hearing individuals, and both
agencies offered TA on using AT. The DRCs also trained JSWDs to use JAWS to attend AJC
workshops. In addition, the AJC career seminar and customer flow were transformed to be
universally accessible, and the disability services planning chart was changed to reflect that of a
typical jobseeker.
Another treatment WDA similarly mainstream[ed] JSWDs through all employment
processes. The DRC from this WDA suggested that CP completion and placement rates for DEI
clients were high because their journeys virtually mimicked the employment processes of WIOA
Adult clients. To facilitate this integration, the DRC conducted staff trainings and served as a
co-case manager for JSWDs to model to staff how to secure accommodations and other
employment-related disability services so customers were suitable for training.” Co-
management of cases also facilitated integration of DEI practices into WIOA services.
This site also tailored multiple workshops for persons with intellectual and learning
disabilities. In terms of accommodations, the DRC arranged for a large keyboard and
magnifiers/readers in every AJC room with a computer and secured captioned telephones. This
AJC also installed adjustable desks and tables and lowered bathroom mirrors. The AJC did not
procure JAWS, however, based on a recommendation from MCB. The DRC also expressed a
desire to have AT to assist people with learning disabilities like dyslexia.
Another treatment WDA featured partners that illustrated universal design enhancements,
as well as an educational partner that was convenient to public transportation and offered
inclusive, student-centered trainings. Its instruction was hands-on, and JSWDs were provided
much testing preparation and could audit classes. Job coaches also spent extra time with anxious
trainees, many of whom had been employed for many years. The third treatment WDA involved
trainers who supplied “extra information” and allowed JSWDs to observe classrooms to be more
comfortable. Low-vision individuals were sent workshop handouts so they could absorb the
information with JAWS prior to the workshop. With reference to DRC and staff capacity to serve
JSWDs, the competitive rebidding of AJC management and staff turnover led to a delay in
implementation and presented obstacles throughout. Two treatment WDAs transitioned to new
operators during Round 5, and the other WDA saw AJCs close, staff laid-off, and “essential
workshops” cut.
Another treatment AJC referred Ticket customers for assignment after intake. In the third
treatment WDA, there were not many Ticket-eligible customers among enrollees, and those who
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
66
were eligible already assigned their Tickets. Moreover, this WDA did not have sufficient staff
capacity to administer TTW, including due to turnover.
Locally, one DRC led whole-staff training all the time,” such as by presenting about
accommodations at staff meetings. An AJC manager credited the DRC with improving disability
etiquette among staff. To accomplish this, the DRC was aided by a train-the-trainer session with
WWL. In another treatment WDA, a DRC joined the grant midway and received a bevy of
training from NDI, ICI, WWL, and the BLN to get up to speed. She also attended trainings on
the Help Wanted Online Index and TORQ systems, which facilitated advanced job search
techniques that prioritized labor market information. AJC staff were also trained on nascent
assistive technologies. The third treatment WDA arranged for full-day, all-staff trainings devoted
to engaging JSWDs. This WDA’s AJC held joint trainings delivered by ICI and another local
provider, including on cultural competence. Upon transitioning to a new operator, staff were
trained on serving JSWDs.
Training to buttress capacity-building was supported by the state’s rich TA infrastructure,
exemplified by ICI and WWL. Early in the grant period, ICI and WWL conducted trainings on
serving JSWDs, TTW, and employer engagement. ICI and WWL also provided targeted
assistance to sites on accommodations, employment placement and supports, self-disclosure,
motivational interviewing, career assessment and pathways, universal design, and disability
awareness. NDI webinars and tutelage were also pivotal for DRCs to learn about the world of
workforce development.”
Treatment WDAs prioritized customer choice, but they also stressed pragmatic career
planning. One DRC submitted that “everything needs to be person-centered.” This AJC
completed interest and skills inventories with each client, and the DRC met with JSWDs
individually. If a client decided to undergo training, he or she typically took the TABE, which
was used diagnostically:
It’s not the score you get that really matters … but you’ll know whether or not you could finish
a training, or that you’ll struggle ... so what can we do to help make your skills work for you so
that you can be able to go?AJC Staff
In another treatment WDA, the AJC utilized CE strategies to develop paid work
experiences that reflected each individual’s abilities and goals. The DRC also partnered with
WWL to help JSWDs envision opportunities “out of their comfort zone.” A JSWD confirmed he
definitely” led his employment search with the AJC. The DRC “constantly” asked him which
opportunities he “could and wanted to pursue,” including his fields of preference and willingness
to travel. The DRC also regularly solicited his “feedback on whether we were going in the right
direction and reminded me I should be driving the search for my own employment.” He never
felt his hand was being held.”
The third treatment WDA also centered on JSWD strengths, interests, and personality. At
the same time, plans had to be goal-oriented,” prioritized, and geared toward addressing
barriers. An individual career plan was formulated based on detailed assessments, which
included identifying preferences in work setting (e.g., in a nursing home or hospital), type of
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
67
occupation, and work schedule. DRCs stressed client commitment and a realistic evaluation of
what outcomes clients could achieve; for instance, those who could not “comprehend a patient
care planwere not a good match” for the CNA profession. Another DRC from this WDA met
frequently with a business services representative to discuss whether client profiles would match
available job opportunities.
A DRC and community-based provider were proactive about accommodations during the
assessment phase: We believe anybody can get a job. How they get there is different, and some
people need more significant supports.” JSWDs with minimal work experience were especially
encouraged to self-disclose because it makes the job coaching when they first start working
much easier to negotiate.” As with most grantees, there was scant evidence of JSWD
involvement in strategic planning. Nevertheless, a WDA director and DEI leader divulged being
hard of hearing, “nearly blind,” and diagnosed with diabetes and a heart condition.
DRCs balanced case management with coordination and collaboration activities. Some
saw case management as their main responsibility, but they also made a concerted effort to
understand and forge systemic coordination. One DRC who joined the grant midway initially
saw it asvery overwhelming. … There wasn’t a full-time person so I just came in and learned
all of the pieces.” Over time, however, she cultivated partnerships and built the AJC’s capacity to
serve JSWDs. Capacity-building was expedited by all-day staff trainings on serving JSWDs and
other DRC functions, including via the “DRC I” training, some of which were open to staff
beyond the AJC. During our site visit, the DRC reported that many JSWDs were served and
placed without her intervention. At the same time, she advised that a DRC role should be
permanently funded at 75 percent full-time equivalent to sustain expertise, partnerships, and
accommodations forged during DEI. In a post-grant call, leadership from this WDA asserted that
cross-training and IRTs led to other staff internalizing how to leverage resources to serve
JSWDs. These leaders also approached DEI as a lever for systemic change, such as around
accessibility. In another treatment WDA, the DRC approached her role with an “exit strategy
and strove to make “really sustainable changes.”
DEI and its employer partners provided child care, work uniforms, and a phone that
JSWDs used to communicate with employers. Employers supplied other employment support
services, including practice with alternative interviewing styles and strategies to help JSWDs be
more at ease with selling themselves.” JSWDs were prepared for OJT scenarios, including
through coping strategies.
34
Employer engagement was integral to DEI implementation across treatment WDAs, with
WWL and business services as key linchpins. WWL facilitated several employer outreach
functions, helped identify employers hiring JSWDs, and arranged for JSWDs to tour employers
through events like Disability Mentoring Day. WWL’s annual career fair featured dozens of
employers and was well attended. Additionally, WWL invited two DRCs to a BLN meeting to
promote DEI and job candidates. JSWDs could also access WWL’s virtual employer bank and
assisted with placement and sensitivity training for employers. Business services staff earmarked
34
Centre for Studies on Human Stress. (n.d.). Coping strategies. Retrieved from https://humanstress.ca/stress/trick-
your-stress/steps-to-instant-stress-management/
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
68
and engaged industry sector employers throughout treatment WDAs and were also central to
navigating turnover within employer partners.
One treatment WDA initially invested much time in employer outreach since there was
limited engagement there prior to DEI and WWL’s BLN was less established there. This WDA
focused on engaging businesses in manufacturing, health care, hospitality, and banking. WDA
leaders also engaged a manufacturing staffing agency to explore JSWDs participating in on-site
trainings and paid work experiences. The agency’s director encouraged JSWDs to apply for the
many local openings, especially since new transportation options emerged to a local
manufacturing site. While the site typically only offered 12-hour shifts, it indicated it would
consider splitting shifts for the right candidates.
DRCs engaged employers in various ways, such as inviting employers to the AJC to
conduct mock interviews; attending career fairs, human resources events, and a WWL-sponsored
BLN event; taking company tours, including of a manufacturing facility; and meeting with
employers to deliver résumés and discuss staffing needs and candidates. A DRC also met with a
local economic development director to connect with employers. Additionally, she arranged for
MCB staff to meet employers and train them about accommodations and to visit and assess work
sites. The WDA chairwoman and business services also engaged employers, including through
presentations at career fairs and trainings.
One JSWD suggested that DEI funding for employment-related internships was helpful
in persuading employers to hire JSWDs. However, he also indicated he could benefit from more
assistance in reaching out to employers and leveraging his own connections; the DRC primarily
sent him online applications for openings. Despite these employer engagement efforts, the DRC
argued that employer bias remained a significant obstacle, particularly if an individual had a
visible disability or was deaf or blind: “I think it’s a huge challenge to get some employers to see
the person and not the disability.” Another hurdle was churn and restructuring within employer
partners; persistence and constant networking were critical to overcoming these transitions.
In another treatment WDA, the DRC worked “really stronglywith WWL to incubate
employer bonds. This synergy resulted in a robust partnership with a local hospital, which led to
many internships and placements. This AJC’s operations manager asserted that the DRC knew
local employers well, worked closely with job developers, and was significantly involved with
job placements. In addition, a job placement specialist from VR ran a regional employment
collaborative. The collaborative met monthly, providing a venue for JSWDs to network with
employers and employment service providers.
In the third treatment WDA, WWL and business services spearheaded employer
engagement. This partnership coordinated multiple job fairs for JSWDs, including for health and
human services and hospitality occupations, as well as one-on-one meetings with employers
about candidates. Like in another treatment WDA, DEI in concert with WWL facilitated a
relationship with a local medical facility that led to a fruitful Career Pathways pipeline. DEI also
helped cultivate links with many other employers, such as nursing homes and Sodexo, which
provides facilities management and food services for schools, universities, and hospitals.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
69
A provider and lead partner in this WDA interacted with employers in furnishing
placement support” services, which included navigating disclosure, résumé development and
distribution, setting up interviews, accommodations, and job coaching. The provider encouraged
employers to allow its clients to attend regular orientation” without accommodations, but it
also conductedinformational interviews to help clients practice seeking accommodations.
These interviews were particularly useful for youth; the DRC argued youth may be less inclined
to engage in self-advocacy with employers since they increasingly grow up in inclusive and
accommodating settings: That’s a big awakening. … They don’t identify as having a disability,
but some of them definitely will need extra supports at work. Most of the time when they fail it’s
because they didn’t want to self-disclose.”
Once a client began working, employers were urged to provide specific feedback on
challenges and clients were observed in action to customize job coaching. Stakeholders also
discussed how it was crucial to understand employers’ perspectives on accommodations,
including allaying fears about confidentiality and liability and gauging their openness to on-site
job coaching. One employer accommodated a trauma-affected JSWD by enabling her to work
near child care. This WDA also leveraged job coaching funding from statewide employment
services to furnish on-site employment supports for two autistic clients working at Sodexo. In
each treatment WDA, leading employers were central to long-term strategic planning for
pathways customized to JSWDs. One employer partner and BLN member was a vocal
advocate” for businesses to collaborate with DEI. The state’s overall proficiency with engaging
employers was evidenced by its leading development of a business engagement Community of
Practice presentation for NDI in concert with WWL.
Career Pathways v.
Two treatment WDAs demonstrated elements of CE that helped individuals enroll in CP
training. A provider and lead partner in one WDA were very active as liaisons between JSWDs
and employers. This included customized job coaching based on employee needs, employer
preferences regarding work opportunities, and the workplace setting. Employer suggestions for
task reassignment were common, and the provider adapted its coaching to these preferences.
Another treatment WDA explored flexible scheduling to enable JSWDs to apply at a
manufacturer that used 12-hour shifts. This WDA also customized an internship for one
individual at a café so that he could work and play music at an “alternativeplace with a small
community feeling.”
In another treatment WDA, the DRC described cost-sharing as the best way to
encompass all of the services that an individual may need.She blended and braided funds with
MCB to prolong internships, as well as to assist with supports like clothing and obtaining a
driver’s license. The third treatment WDA leveraged resources from MRC and the Department
of Developmental Services. This enabled it to offer more individualized retention services,
including extending onboarding and providing transportation supports. The lead partner and
provider also engaged local colleges to seek funding for individualized supports that would
ensure that individuals had access to CP training. For example, a JSWD whose reading is not at
an eighth-grade level will likely not flourish in a patient access and registration class. To upgrade
basic skills, a DEI AJC offered a “Career Ready 101” online curriculum. The AJC was proactive
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
70
in identifying possible accommodations. Despite these tactics, treatment WDAs still faced
obstacles identifying JSWDs who could commit to and complete CP training. Challenges
included chronic health problems, such as terminal and mental illnesses, ex-offender issues,”
and refugee statuses. ICI and WWL were consulted to address these hurdles and to help identify
suitable candidates. Disclosure was another challenge, especially for those with mental health
disabilities.
A DRC contrasted DEI with a prior system that funneled JSWDs to VR rather than
trying to really figure out what the JSWD actually needs … Maybe they’ve been working with
VR and it’s not been a good process.” The AJC also had much more access to employers than
VR, could intervene early in the job search process, and was not hamstrung by the wait time or
inflexibility that characterizes VR: You better go to VR knowing what you want … because
they’re not going to open up the menu of services and tell you, ‘You have this option or this
option.” The DRC also distinguished the AJC from SSA’s customer service and suggested it is
crucial to assist JSWDs so they can avoid SSA’s disincentives to work.”
JSWDs corroborated that the AJC and DRC balanced customer choice with realistic
options. One was transformed from feeling lostto learning about stackable credentials and
studying the local labor market and projected growth occupations to match with her skills,
background, and disability. She also stated that JSWDs “have to put some work into it too.”
Another individual relayed how the DRC built upon her desire to be a nurse by encouraging her
to pursue a medical internship, which opened her to “a new world” and “a dream come true.”
After her internship, she secured a full-time temporary job. The DRC considered the JSWD’s
aspirations and abilities. She was “always positiveand offered sound step-by-step advice. This
JSWD deemed the AJC as very helpful, including its career fairs, and referred her peers to the
AJC. A third participant attested that the DRC was “ready to help me with everything I need” and
helped her secure a medical internship.
With respect to Career Pathways, treatment WDAs focused on health care (e.g., CNA,
medical assistant, home health aide, sterile processing assistant, surgical technology, and central
processing), culinary arts, hospitality, and manufacturing (e.g., machinists). Hospitality trainings
were generally of shorter duration and led to quicker placements.
DRCs facilitated various customized assessments and utilized state-sponsored CP tools
and O*NET OnLine. AJC enrollees also had to complete the state’s mandatory Career Center
Seminar. One DRC conveyed how CP-specific assessments could shape a JSWD’s employment
journey. For instance, those with developmental disabilities could struggle to pass the entrance
examination for CNA trainings; following directions sequentially and being able to comprehend
a patient care plan are crucial to being a CNA.
For the lead partner and provider in this WDA, individualized assessment was essential to
identifying appropriate supports. Assessments occurred in a comfortable setting: We have their
assessments, so they can do it untimed in a relaxed place.” Assessments gauged strengths and
barriers to employment, informed strategies to tackle barriers, and determined whether assistance
was needed to get fully trained.” After a series of technical and readiness assessments,
individualized supports were developed. A “commitment assessment” helped determine how
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
71
equipped a jobseeker was to enter a pathway. Assessment also honed in on the client’s ideal
work situation:
Are they going to want to be working in a nursing home or in a hospital? Do they want to work
in a home-based program as a home health aide? If they’re doing culinary arts, are they
interested in working in the cafeteria? Do you want to work in a restaurant? Do you want to
work Monday through Friday?DRC
This provider also delivered 30 hours of job readiness instruction before clients enrolled
in CP. Training focused on soft skills like time management, professional etiquette (e.g.,
documenting absences and workplace communication), socializing at work, and dress codes.
This training period also allowed clients to obtain “extra supports,” such as for transportation.
In another treatment WDA, the DRC used various assessments to “see where they are,
where they’re coming from, and where they want to be,” including the Massachusetts Career
Information System, Holland Self-Directed Search, the Work Readiness Assessment, and TORQ.
The DRC also arranged career development coaches to help calm fears” about starting a course
or taking a test.
The third treatment AJC’s assessment process included a Career Directions workshop
that offered interest and skills inventories, as well as TABE testing. During pathway trainings,
DRCs helped request academic supports at community colleges, such as one-on-one tutoring and
group sessions to prepare for certification exams. One DRC also advocated for participants to
retake exams multiple times. Another DRC attended information sessions at community colleges
with JSWDs to help them feel more comfortable with college staff. This WDA also met with a
college’s coordinator of disability services, services that included accommodations (e.g., note-
taking) and services provided by the school’s “Visions
35
program. This meeting led to other
connections between the AJC and the college, and some Visions attendees were trained through
DEI.
One treatment WDA nurtured a fruitful partnership with a local hospital, which led to
numerous successful internships, paid work experiences, and placements in patient access and
registration. According to multiple data points, “nearly anyone” who completed an internship
with this hospital quickly landed a job. Moreover, these positions were often full-time and long-
term. The DEI State Lead indicated that two DEI graduates of this training were offered full-time
jobs starting at $16.67 per hour; both were long unemployed and earned just above minimum
wage in their last jobs. One of the individuals was a TTW beneficiary.
One WDA did not have a prosperous relationship with the CVS internship program. DEI
funded retail and customer service internships through CVS. At least six participants graduated
from the program, and CVS provided positive feedback, yet none were hired. The DRC and a
partner opted to end the partnership with CVS. Nevertheless, this WDA exceeded grant
objectives according to the DEI State Lead, as they achieved success with multiple TTW clients.
35
Visions supports disadvantaged students to persist through graduation and helps with transferring to 4-year
college institutions. The program provides a variety of services, supports, and accommodations.
Disability Employment Initiative Evaluation
Social Dynamics, LLC/Abt Associates
72
One gained an associate’s degree, finished a nursing program, and was considering becoming a
registered nurse. She was in the interview process and withdrew from SSA and SNAP benefits.
In another treatment WDA, DRCs help JSWDs focus on “the end productfrom the
beginning; “everything we do has to be employment-focused.” With DEI, this WDA built upon a
grant from the state’s Commonwealth Corporation to continue a partnership with a local
community college. The state grant focused on CNA, an allied health, and culinary arts
pathways. This WDA also coordinated a summer boot camp for youth interested in culinary arts
or CNA pathways. The camp focused on work readiness, such as résumé writing, networking,
interviewing, and learning styles.
DEI and WWL helped this WDA develop its CP infrastructure further during Round 5.
WWL facilitated a meeting with a local rehabilitation hospital