Project Overview - crwg-gdrc.ca€¦ · Web viewProject Overview. Common Indicators: Transforming the Culture of Evaluation in Career and Employment Services

1

2

Project OverviewCommon Indicators: Transforming the Culture of

Evaluation in Career and Employment Services

In May, 2012, Human Resources and Skills Development Canada (HRSDC) launched a national research agenda to better understand “what works” in career and employment services. This project is 1 of 3 that were approved under this initiative. It is an 11 month project focused on testing a comprehensive evaluation model that tracks:

changes in clients as a result of programs and services (new knowledge acquired, new skills acquired, changes in personal attributes such as self esteem and ability to self manage);

traditional client outcomes of employment and entrance into training or educational programs; quality issues related to employment or training such as the fit of employment with client skills

and qualifications, and fit of training with client preferred future vision; and client employability issues and challenges as well as different employment opportunities in

different locations.The evaluation model has been developed over several years by the Canadian Research Working Group for Evidence Based Practice in Career Development (CRWG). This group, established in 2001, is a partnership among 8 researchers from both Anglophone and Francophone universities and the Canadian Career Development Foundation (CCDF). Depending on the project, all or some of the researchers take lead roles along with CCDF consultants.

The project team for this project is comprised of: The Canadian Career Development Foundation (CCDF) (Lynne Bezanson & Céline Renald); Dr. Guylaine Michaud (Lead Francophone Researcher, Université de Sherbrooke); Dr. Dave Redekopp (Lead Anglophone Researcher, Life-Role Development Group, Edmonton,

Alberta); Dr. Reginald Savard (Université de Sherbrooke) and Dr. Liette Goyer (Université Laval); Dr. Bill Borgen (University of British Columbia); Ken Organ of Goss Gilroy Inc. (GGI), a nationally recognized firm known for its expertise in

evaluation and the development of accountability frameworks; and YOU – Career/Employment offices across Saskatchewan, Québec and New Brunswick who have

volunteered to participate as part of the research team. Many of you have already helped to shape the evaluation model by participating in focus groups, which took place in June, 2012. You will be leading recruitment, field testing the model, and gathering all critical data for analysis. We hope this will be a learning opportunity for you as we work together, and we also know that we will learn immensely from your experience and your feedback. Together we believe that this project has the potential to make a significant contribution to the field—but of course we need to test it to find out if we are right!

3

THE PROJECT IN A NUTSHELLThe project will develop and test an online tool that will continue to gather traditional indicators including employment and training, include the newer indicators outlined above, and go further to include additional indicators and outcomes not yet tested. The indicators will be based on results of previous CRWG research, input from the literature review and focus groups with front line practitioners and managers who will be partners in the research itself.

All clients who are registered to receive services from a career development practitioner who is participating in the research will be eligible to participate in the research. There are no recruitment criteria.

For each client, participating practitioners will conduct a formal employability needs assessment as is normal practice. If the client is going to receive ongoing service from the practitioner (that is, will be seen more than for a single assessment interview), the practitioner will invite the client to participate in the research.

All participating clients must sign a consent form and will be asked to complete short questionnaires at the beginning and at the end of the study.

Participating practitioners will use the online tool to record client data, results of the needs assessment interviews, the kinds and duration of services given to the client, and incremental changes as appropriate after every client contact.

Participating practitioners will also continue to use the data collection system currently in use in their offices. This will require entering some data twice and will necessitate extra time being given to participating practitioners. As we are field testing a new system, we recognize that it is unlikely to be perfect and offices cannot compromise their ongoing data collection responsibilities. These details will be fully negotiated with office managers who are highly supportive of the research study.

The duration of the research will be 10 weeks (January to March 31, 2013); Follow-up interviews will be conducted with participating practitioners and managers to explore

their experience using the online tool, identify any anomalies and gather recommendations; Participating practitioners will also be asked to complete a short questionnaire at the end of the

study. Clients who participate in the research and complete the questionnaires will receive a small

honorarium or gift card equivalent ($20.00) in appreciation. A full report will be prepared on the research, its results, what has been learned, the implications

for service delivery and recommended next steps.

Research QuestionsThe research questions to be addressed are:

1. What common indicators are applicable across different client contexts, different client groups, different agencies, and different interventions?

2. What statements can be made about service effectiveness by tracking common indicators of inputs, processes and outcomes? If numbers permit, it will be possible to aggregate data to begin to address the ultimate question, “What kinds of interventions in what contexts produce what kinds of outcomes?”

4

Project Objectives1. Identify an agreed upon set of common input, process, and outcome indicators to be collected

across agencies;2. Develop, field test, and implement an online data management tool (English and French) that

merges agreed upon common indicators with the CRWG evaluation framework;3. Revise and refine the data management tool in accordance with the results of the

experimentation and the experience of end-users; 4. Report on the connections among Inputs, Processes and Outcomes for diverse client

employability profiles, specific groups with low labour market attachment and divergent opportunity structures; and

5. Establish broader implementation plans with participating agencies in order for them to integrate the data management tool into their existing evaluation systems and evaluation into the everyday culture of their services on an ongoing basis.

Anticipated LearningThe tools and processes to be developed and tested in this research will respond directly to the demands from the career practitioner field. For many years, practitioners have asked for more robust data-gathering tools that capture more fully and accurately the results achieved with clients. Employment, reduced time on benefits and successful retraining are currently measured but, with very few exceptions, other variables influencing success are not: skill acquisition; increases in confidence and optimism; motivation to pursue an employability goal; qualitative measures related to how well employment and/or training fits with individual ambitions and qualifications; and the actual quality of the employment itself.

The tools and processes will capture the growing complexity of career and employment services practices and outcomes and take into account factors that influence and, in many cases, severely limit and/or enhance individual choice. The tools will begin to map learning outcomes onto interventions and recognize in practice that “one size does not fit all.”

Because of funding requirements, the research must be completed by March 31, 2013. It is recognized in advance that a period of ten weeks is very short to be able to capture changes in knowledge, skill, and attitude as well as sustainable employment and long-term savings to benefit programs. However the researchers are convinced that it is long enough to begin to see trends in client change. If this proves true, we will be on solid ground to seek support for longer-term experimentation and implementation.

The tools and processes developed in this project will not answer all demands for more robust evidence, but will move the evidence base significantly forward. The common indices data to be gathered will include a considerably broadened range of outcomes, all of which will measure levels of client change. Participants in this research will be making an important contribution to building a stronger and more holistic culture of evaluation in career and employment services.

5

6

7

Employability Assessment InterviewThe U-Name It tool is designed to capture the amount of client change which occurs as a result of career and employment services received over a period of time. In order to do this, the system needs to have a baseline of where the client is at the beginning of receiving services. This baseline will allow us to capture both client progress and client outcomes at the end of the research period.

Therefore the Assessment or Intake interview is very important to the study. We recognize that practitioners have different styles for conducting an assessment interview and, rather than provide a script, we have identified the factors that should be explored in order to assess a client’s employability needs and tailor services and programs directly to that client’s needs. We have also included an Assessment Checklist as a tool for your personal use should you find it useful.

The study is open to all clients who are going to see a practitioner more than once – that is all active clients. Existing as well as new clients are welcome into the study. In all cases however a baseline is needed so an Assessment Interview capturing where the client is NOW at time of entry into the study, not at time of entry into services, is essential. As long as the system has a client starting point or baseline, it will be able to track client change.

An Assessment/Intake Interview is the entry point into the research.

8

Protocol for Employability Assessment Interview

This is not a protocol in the strict sense of the word. We recognize that practitioners have different styles for conducting an interview and, rather than provide a script, we have identified the factors that should be explored in order to assess a client’s employability needs and tailor services and programs directly to that client’s needs. Generally speaking, the interview will follow the three steps below and address the content itemized in the “Points to Help in Determining Need” chart as well as the “Checklist for Employment Assessment Interview,” both in this section.

Step 1: Establish A Collaborative Relationship Greet client and determine client objective Establish the purpose of the interview Confirm motivation and client commitment

Step 2: Gather employability information Gather data on employability of the client

o Job-readinesso Self-awareness and career/employment direction/goalo Ability to look for worko Meeting occupational requirements/getting qualifiedo Ability to maintain work

Verify client perspective Obtain agreement on identified needs

Step 3: Develop an action plan Come to agreement on a goal to be achieved Come to agreement on what “success” for the client at the end of your work

together will look like Come to agreement on first steps to take towards the goal Check client commitment Establish follow-up plans Close interview

9

Employability Dimensions

Client employability needs are organized into five broad employability dimensions: Pre-Employability/Job Readiness; Career Decision Making; Work Search; Skill Enhancement and Employment Maintenance. Each employability dimension lists a number of indicators which suggest specific needs within each dimension. No list of indicators can be complete; there must always be room for an “Other” category to capture the diversity of client needs. In general however, within career and employment services, needs fall into the following general dimensions:

PRE-EMPLOYABILITY/JOB READINESS: Has the intention to seek employment and/or enter training but faces challenges, vulnerabilities and/or skill deficits that need to be addressed before the client can be successful in achieving employment/training goals

CAREER DECISION MAKING: Not yet clear about work/learning goals; needs to understand personal skills, interest, values and personality and more about different potential occupations in order to develop a vision for a preferred future

SKILLS ENHANCEMENT: Has career goal, but needs to explore skills and qualifications needed for the work they want and how to acquire them

WORK SEARCH: Has career goal and necessary skills/qualifications for desired work; needs skills, strategies and tools necessary to conduct a successful job search

EMPLOYMENT MAINTENANCE: Has difficulty keeping work or may need to work on the skills/attitudes required for success at work

OTHER: Many circumstances beyond those covered by the employability dimensions may impact on the client’s employability outcomes. Some of these life circumstances may necessitate referral outside of career and employment services. Life circumstance issues may need to be addressed and resolved before progress toward employment can realistically be made.

10

-MAKING

SKILLS ENHANCEMENTJOB MAINTENANCE

Points to Help in Determining Need

Pre-Employability/Job-Readiness

Client needs assistance to:

identify and clarify future direction (training, education , employment or change in life circumstances goal)

identify personal strengths/resources that support future direction (training, education, employment or change in life circumstances goal)

resolve specific financial challenges/vulnerabilities that may impact on future direction (mortgage, public transit, childcare etc.)

access community resources/supports that address specific personal challenges/vulnerabilities and assist in helping move goal forward (mental health services; addictions counseling; public housing etc.)

develop necessary personal supports needed to move toward future direction (family, childcare, transportation)

acquire life/employment skills which support future direction acquire attitudes which support future direction Develop and follow a plan of action to move future direction

forward

Career Decision Making


identify own strengths, skills, interests, values, transferable skills connect strengths, skills, interests, values, transferable skills to

career choices research work opportunities for work using a range of sources (job

boards, labour market information, internet, networks, employer and employee contacts)

research details specific to career goal (time needed in education/training; future employment prospects; types of work; places of work; local opportunities)

identify a career goal (training, education , employment or change in life circumstances)

identify personal resources that support achievement of career goal (support system, finances, motivation)

identify challenges, internal or external, which may interfere with achievement of career goal (mobility, local opportunities, finances, health)

develop and follow a plan of action to mitigate challenges and move toward career goal

11

Work Search Client needs assistance to:

confirm employment goal confirm that qualification and experience are consistent with

employment goal identify personal strengths that support successful work search identify potential employers and potential employment

opportunities adjust/adapt employment goal with employment opportunities as

needed identify transferable skills complete appropriate resume and cover letter use networks to identify employment leads use resources, tools and methods to support work search

including internet tailor resume and cover letter according to work possibilities demonstrate appropriate job interview skills develop and follow a plan of action for active work search demonstrate work attitudes and behaviours to support successful

job search be active and persistent in work search

Skills Enhancement


confirm training/education goal research future employment prospects related to

training/education goal/program prior to pursuing education/training

research training/education options available to achieve training/education goal

evaluate and choose training/education options taking into consideration personal circumstances (supports, strengths and limitations)

identify potential challenges that may impact on achievement of training/education goal

proactively develop strategies to address identified challenges acquire study and personal skills needed to be successful in

education/training identify resources that are available in training and education

sites and/or community agencies to provide help and guidance supporting successful completion of program

sustain motivation to complete training/education program

Employment Client needs assistance to:

12

Maintenance identify important skills and attitudes that improve chances of keeping employment

identify personal strengths and limitations with respect to these skills and attitudes

develop a learning plan to acquire/strengthen skills and attitudes before on the job problems arise

identify community and/or workplace resources that provide help and guidance related to keeping work

know job roles, responsibilities and expectations that support being successful on the job (who to report to; who makes decisions; approval processes, getting answers to job reacted questions)

assertively seek assistance when needed develop and follow a plan to remain up to date with on the job

changes in duties and competencies

Additional Life Circumstances

improved housing is needed improved transportation is needed improved capacity to work and/or study is needed Increase in sense of responsibility for own choices and behaviours

is needed Increase in ability to set short and long term goals is needed Increase in understanding expectation and demands of employers

is needed Reduction in destructive behavior is needed Following medical and medication protocols is needed Increased openness to change is needed Increased access to constructive and positive support systems is

needed Improved relationships with family and friends are needed Increased trust in other people is needed

Personal Attribute Needs


Develop stronger self-management skills (able to take steps forward independently; open to using available resources; stay motivated to follow steps in action plan)

Develop stronger self-esteem (optimism; confidence; sense of self as competent)

Develop stronger sense of personal well-being (good energy to pursue goal; willing to use support systems and available resources; open to learning and suggestions)

Develop stronger self-efficacy (belief in own capacity to overcome challenges; confident in a positive outcome despite obstacles)

13

Checklist for Employability Assessment Interview: You may find this a useful checklist to keep you on track in your Employability Assessment Interviews and to confirm that you have not omitted some important areas for exploration. Many professions (surgeons; pilots) use checklists constantly and they have been proven to reduce mistakes and errors. It is not required that you complete the checklist; it is for your own professional growth.

Checklist for Employability Assessment Interview

Client:

Research Practitioner:

STEPS Not done

Sort of done

Done

STEP 1: Establish a Collaborative Relationship

Greet client and explore client objective

Establish the purpose of the interview

Establish a climate of trust and comfort in working together

Invite client to actively participate in the interview

Confirm client’s motivation and commitment to objective

STEP 2: Gather Employability Information

Gather data on pre-employability/job readiness issues:

Client’s career/employment direction/vision/goal

Client’s self-awareness (personal strengths, interests, abilities)

Client’s financial challenges/vulnerabilities (mortgage, debt, transit, day care etc.)

Access to resources to address personal challenges/vulnerabilities (mental/physical health issues, addictions etc.)

Client’s personal support system (family, transportation, friends)

Client’s life/employment skills

Client’s employment attitudes

Client’s openness to developing and following an action plan to move toward goal achievement

Gather employability information relevant to Career Decision Making:

Client’s self-knowledge (strengths, skills, interests, personal characteristics)

Client’s ability to connect self-knowledge to career options

14

STEPS Not done

Sort of done

Done

Client’s ability to research career options , training and/or work opportunities

Client’s career goal (employment/training/education/change in life circumstance)

Client’s access to personal resources to support goal achievement (support system, finances etc.)

Client’s recognition of challenges that may interfere with goal achievement (finances, health, mobility, local opportunities)

Client’s openness to developing and following an action plan to move toward goal achievement

Gather employability information relevant to Job Search:

Clarity of client’s employment goal

Clarity that qualifications and experience match employment goal

Client’s ability to use personal strengths that support successful job search

Client’s ability to make use of resources to support work search (networks, support systems, websites, job boards)

Client’s ability to identify potential employers and work opportunities

Client’s ability to identify transferable skills

Client’s need for appropriate job search tools (resume, cover letters, references)

Client’s mastery of interviewing skills

Client’s ability to demonstrate effective work attitudes and behaviours to support successful job search

Client’s capacity to sustain motivation and persist in work search

Gather information relevant to Skills Enhancement:

Clarity of client’s training/education goal

Client’s ability to research employment prospects prior to pursuing training/education

Client’s ability to research training/education options to achieve training/employment goal

Client’s ability to evaluate options taking into account personal circumstances (supports, finances, access)

Client’s need to acquire study and personal skills needed to be successful in education/training

Client’s ability to identify resources which provide help and guidance to

15

STEPS Not done

Sort of done

Done

support successful completion of program

Client’s capacity to sustain motivation

Gather Information relevant to Employment Maintenance:

Client’s ability to identify the important attitudes and skills which will improve chances of keeping employment

Client’s ability to identify own strengths and limitations with respect to these skills and attitudes

Client’s knowledge of resources in the community and workplace which provide coaching related to keeping work

Client’s ability to assertively seek assistance when needed (before problems arise)

Client’s understanding of the need to remain up to date with job changes requiring new knowledge and skill

Gather Information related to client’s life circumstances if and as needed

Improved housing needed

Improved transportation needed

Improved capacity to work and/or study needed

Increase in sense of responsibility for own choices and behaviours needed

Increase in ability to set short and long term goals needed

Increase in understanding expectations and demands of employers needed

Reduction in destructive behavior needed

Following medical and medication protocols needed

Increased openness to change needed

Increased access to constructive and positive support systems is needed

Improved relationships with family and friends are needed

Increased trust in other people is needed

STEP 3: Assess Personal Attribute Needs

Client’s self-management skills: (ability to take steps forward independently; open to using available resources; motivation to follow steps in action plan)

Client’s self-esteem: (optimism; confidence; sense of self as competent)

16

STEPS Not done

Sort of done

Done

Client’s sense of personal well-being: (good energy to pursue goal; willingness to use support systems and available resources; openness to learning and suggestions etc.)

Client’s self-efficacy: (belief in own capacity to overcome challenges; confident in a positive outcome despite obstacles)

STEP 4: Develop an Action Plan

Come to agreement on a client which is owned by the client

Come to agreement on first action plan steps to take towards the goal

Check client commitment to action plan

Establish follow-up plans

Close interview

17

Arriving at an agreed upon goal and a concrete action plan are fundamental. They are the roadmap to guide and to gauge progress towards the client achieving his/her employability goal. They must be completed after every client interview. “U Name It” will save all the Action Plans for your client so you and your client can always see what was agreed to, discuss the progress which is being made, and make revisions as needed. There is lots of evidence that telling someone out loud that you will do something and/or letting another see what you have written that you will do motivates action.

Action PlannerGoal

Action steps Do this by Finished(check)

1.

2.

3.

4.

5.

Other Supports for Client (people, programs, services, resources)

Contact Information for Other Supports

18

Invitation to Client to Participate in Research Study

You will need to invite clients to participate in the research and be clear with them about what the research is about, why their participation is important, what is expected of them and what they will get out of participating. A suggested script is provided below. You will want to work with it and put it into your own words (while covering the main points) so that you are comfortable and natural in presenting the research opportunity.

Suggested Script

Our office is participating in a research study to try to better understand the results of our programs and services and how to keep improving them. The study is taking place in employment offices in Saskatchewan, Québec and New Brunswick. The systems we traditionally use gather very basic (but important) information on whether or not our clients got work and whether or not they succeeded in a training program. We would like to know more than this.

We want to gather information such as whether the job was a good match for the skills and qualifications of our clients, whether the training was a good match with the client’s vision of their future work, and if our clients learn new skills and knowledge about how to be successful in work or training from our programs and services. We also want to know if our services helped clients to feel more positive and optimistic about their futures. We think all these factors help to bring about good results but we need to test the model out to see if we really can measure these things.

I would like to invite you to be part of this study. I would like to tell you a little about it so you can decide if you would like to participate. Is that OK?

Here is how the study works.

If you agree to participate, there will be a few forms to complete – a form saying you agree to participate and a questionnaire asking for some background information about you and your career and employment history to date. These should take no more than 10 minutes to complete. I will provide you with the

19

forms and will answer any questions you might have as you complete them or we can complete them together if you prefer.

At the end of our work together or after 6 weeks (whichever comes first) you will again be asked to complete one questionnaire, which will ask about what you learned and where you are now with respect to your own employment or training goals. This should take no more than 15 minutes to complete. Again I will give you the questionnaire and you can complete it here or I can send it to you via e-mail and you can complete it and send it back to me.

In between, you will receive all the regular services and programs that would be offered whether or not you were in the study. I will be the practitioner you work with for the full period of the research study.

If you agree to participate, you will receive a $20.00 gift card as a “thank you” for helping us to understand how to improve our services to clients. Your participation is totally voluntary and you are completely free to leave the research at any time.

The researchers will publish the results of the study but you will not be identified by name in any of the reports on this research and any information you give on the surveys will be confidential and used only for the research.

Are you interested in participating?

If no, thank the client for listening.

If yes, thank the client for agreeing and ask if he/she has any questions before filling out the forms.

20

Consent Form

In order for the research to meet ethical standards, clients must sign a consent form indicating that you have explained the research to them, they understand and are voluntarily agreeing to participate. All Client Consent Forms must be mailed to CCDF . Pre-addressed enveloped have been provided.

A suggested script follows:

Consent Form:The researchers need to confirm that you agreed to participate and that you are very clear about what you are agreeing to. Here is a consent form for you to sign; you may of course keep a copy if you would like to.

It really covers what I just explained to you and it reassures you that your participation will be confidential.

(Quickly walk through each paragraph highlighting its main content – just a few words per paragraph)

Please read it and ask any questions you have….

Are you fine with the form?

Please sign.

If client wants a copy, please make a photocopy; make sure two copies are signed; give one to client; put second one in envelope provided.

21

PARTICIPANT CONSENT FORM

Please read the Informed Consent Information below. By signing your name at the end of this document, you indicate that you have read and understand the nature of your participation in this study, and that you voluntarily agree to participate as follows:

I agree to participate in the study entitled, “Common Indicators: Transforming the Culture of Evaluation in Career and Employment Services”, under the direction of senior researchers Dr. Dave Redekopp and Dr. Guylaine Michaud and the Canadian Career Development Foundation (CCDF).

I understand that the purpose of the study is to gain a better understanding of the results of our programs and services for clients beyond helping people find jobs or helping people to access training programs.

I acknowledge that my participation in this study will consist of:

o Participating in an interview to help identify my needs;

o Signing a consent form and completing an initial questionnaire covering some background information;

o Completing one questionnaire covering my own learning at the end of the research project at the end of my last interview (whichever comes first); and

o Participating in the programs and services as mutually agreed to by me and my career development practitioner.

Upon completion of the final questionnaire, I understand that I will receive a gift card valued at $20.00.

I understand that the information I provide will be provided to the researchers and will be kept confidential. I also understand that the analysis of the data will be included in a report of the study findings that may be presented at meetings and professional conferences, or published in professional publications, newsletters and academic journals.

I am confident in the assurances provided by the researchers that the information I share will remain strictly confidential and that:

o answers I provide will be stored in a secure location on a computer; and

o only the research team will have access to the consent form and actual data.

I am aware that I may keep a copy of this Informed Consent form for my records.

22

I understand that if I have any questions about the conduct of the study, I may contact the Office Manager who is responsible for this study or Lynne Bezanson, Executive Director, Canadian Career Development Foundation, (613) 729-6164, ext. 204 OR [email protected].

I have read the above material, understand the nature of my participation in this study, and I voluntarily agree to participate.

Yes, I agree to the above statement.

Name (please print):

Signature:

Date:

23

mailto:[email protected]

RESEARCH TIMETABLE AND RECRUITMENT TARGETS

The research will run from January 14 until March 29, that is a period of 11 weeks in total.

You are asked to use U NAME IT for a period of EIGHT weeks for each client you recruit into the study. Week ONE will be the Assessment week followed by SIX weeks of regular services followed by ONE week for completion of the EXIT interview (from the research, not necessarily your caseload).

The following timetable will be a guide:

RECRUITMENT INTO THE RESEARCHASSESSMENT INTERVIEW

PROVISION OF REGULAR SERVICES

EXIT FROM THE RESEARCH

CLIENT QUESTIONNAIRE COMPLETION AND CLIENT ACKNOWLEDGEMENT

JANUARY 21-25 JANUARY 28-MARCH 8 MARCH 11-15 MARCH 11-15JANUARY 28-FEBRUARY 1

FEBRUARY 8-MARCH 15

MARCH 18-22 MARCH 18-22

FEBRUARY 4-8 FEBRUARY 11-MARCH 22


RECRUITMENT TARGETS:

Ideally we would like to have a minimum of 300 clients recruited into the study. Given that these can be existing or new clients, we hope that is not too ambitious.

In Saskatchewan we have 39 participating practitioners which translates into each practitioner recruiting each week THREE clients into the study for a total of NINE clients per counsellor over the course of the research. Again these can be existing clients who agree to be part of the study as well as new clients. The more recruited into the study, the better for data analysis so there is no upper limit.

We have included a Record of Client Intake which you may find useful in order to keep track of your own recruitment numbers and the scheduling of exit for interview dates (See Part 4). The U Name It system will have this information as well but you may find it helpful to have a backup system to remind you.

We will be asking your office managers to do a weekly count of the numbers of clients recruited into the study. We will also ask them to collect the Client Consent Forms and send them to us weekly as well as the Final Client Questionnaires. Weekly teleconferences will be held with all managers and any difficulties you are experiencing or questions you have will be addressed at that time. Managers have access to researchers at all times and they are your conduit to the researchers for immediate resolution of any issues. You also have a 1-800 number for any technical issues which is available to you during all working hours. The number is 1-800-289-1407.

24

THE ROLES OF RESEARCH PARTNERS PRACTITIONERS AND MANAGERS

PRACTITIONERS: KEEPING A LOG:

In addition to recruiting clients into the study and following the research protocols, we want to gather feedback from front line providers regarding the U Name It tool, its usefulness, ease or difficulty of use, identification of problems and suggestions for improvement. We know that U NAME IT will not be perfect in its first trial run and we know that the best way to improve it is to have the feedback of front line practitioners who are using the system. We want to, over time, create a robust and holistic tool which captures real client change and demonstrates the real value of career and employment services.

You can help us by keeping a log over the course of the research of what worked well, what did not work well, what you wanted to swear at, and what suggestions you have to make it better. A log format is included (See Part 4) and we would be most grateful if you would save this on your own computer, complete it periodically throughout the research and send it to us so we can continue to improve the tool. Please send to [email protected] as often as you wish. We are encouraging you to do this log at least weekly.

MANAGERS: KEEPING THE RESEARCH SMOOTH:

Practitioners who are active in the research will have to perform double duty with respect to record keeping. In addition to your own regular office system requirements, they will be using the U Name It system to record client information. It is difficult to estimate the exact extra time needed but we suggest that 15 additional minutes be provided to practitioners in the study for each interview with a client who is in the research. This will require some internal adjustments and perhaps practitioners not in the study agreeing to take responsibility for additional clients for the duration of the research.

We are asking managers to keep close contact with practitioners in the study so that operational adjustments can be made if necessary.

Practitioners and managers will have access to the 1-800 line for technical questions and issues.

In addition, we are asking managers to collect Client Consent Forms weekly and send them to CCDF in the envelopes provided. At the end of the research, there will be a client questionnaire which most clients will complete online. Some however may complete the questionnaires manually and these will need to be sent to CCDF if the envelopes provided. Again, we would ask managers to assume this task.

Finally, we would like to schedule a weekly teleconference call with managers so that any issues related to the research itself can be brought to our attention. This will require that managers ask for feedback from all participating practitioners on a weekly basis. If there are very few or no issues, these calls can be very short. They will also be an opportunity to monitor recruitment progress and any difficulties being encountered in meeting the recruitment targets.

25


26

U NAME ITA paper copy of the on-line tool is included here for your easy reference. You may find it useful to have a paper copy initially but only until you are familiar and comfortable with U NAME IT.

INTRODUCTION:Thank you for agreeing to be part of this “Common Indicators” research project. We are testing out a new data collection tool which we hope more accurately and completely captures the actual outcomes that clients achieve as a result of career and employment programs and services. Some of these outcomes are practical and easy to measure (e.g., did the client get a job?); some outcomes are more related to learning and are more subjective (e.g., did the client learn how to research training courses/how to make a cold call?); some are internal and less easy to measure (e.g., did the client become more self-confident/more independent?). Because some outcomes are harder to measure does not mean that they are less important or that they do not contribute to positive client outcomes. The tool we are testing with you tries to capture all these types of outcomes and it also tries to go further and begin to track what kinds of programs and services appear to result in what types of outcomes.

For the research period, we are calling the tool the “U Name It” system. It needs a good name, descriptive AND friendly but we cannot find one. We hope you can. Send suggestions to us at [email protected] in either French or English (or both!). There will be a prize for the winners, to be determined when we know how much money is left in the budget. Your creativity will be appreciated, as is your participation in developing a tool that we hope will transform how career and employment services are evaluated.

WELCOME TO “U NAME IT”!(January 2013)

27


CLIENT, OPPORTUNITY AND SERVICES CONTEXT:

INPUTS:

EMPLOYMENT OPPORTUNITY STRUCTURE:

U NAME IT: OFFICE INFORMATION: CATCHMENT AREA PROFILE

Management has provided us with information for each participating office. The information gives a picture of local and regional employment, training and community opportunities for clients you serve. You may review this information by clicking on the Catchment Area Profile at any time but you are not required to do so.

1.1 Employment Opportunities locally and regionally

1.2 Training/Education/Upgrading programs available regionally and locally

1.3 Community Resources available locally (daycare; transportation; other)

28

2. PRACTITIONER PROFILE

U NAME IT: USER ACCOUNTS: PRACTITIONER PROFILEYou are asked to complete this the first time you log onto the system by clicking on Practitioner Profile. This information will give a picture of the career practitioners who provide services to clients across the province. Once completed, you will be automatically linked to the research number assigned for the project. You may review the information you have provided at any time but you are asked to input it once only.

2.1 Your age: 15-25 26-30 31-40 41-50

51-60 61-70 71 plus

2.2 Your postal code:

2.3 Your gender:

Female Male

2.4 Years in the employability field:

0-4 5-9 10-14

15-19 20 or more years

2.5 What percentage of your work time is devoted to providing direct career development/employment programs and services to your public?

75-100% 50-75%

25-50% less than 25%

2.6 How did you acquire your knowledge and skill in career development? Choose as many as apply.

Formal education

Formal on-the-job training

Informal on-the-job training (coaching, mentoring, observation)

Professional development off the job (subsidized by my employer)

29

Professional development off the job (self-funded)

Volunteering

Other: (Please specify)

2.7 What is your background training/education/qualifications?

Undergraduate degree(s) in

Undergraduate diploma(s) in

Post-graduate degree(s) in

Post-graduate diploma(s) in

Certificate(s) in

On-the-job training in


2.8 Average number of clients:

per day /per week

2.9 Are you a member of a professional association?

Yes No

30

3. CLIENT EMPLOYMENT POTENTIAL Once clients agree to participate in the research they will be asked to complete a questionnaire. This data will be entered once only. Depending on the practitioner’s personal style, this information may be collected interactively with the client or completed on paper by the client and entered into the system by the practitioner after the initial intake interview. This information will be entered once only.

Client Demographic Information

3.1 Please provide the background information requested below. This information will help us evaluate the effectiveness of the services we are providing. You may omit any questions that you feel uncomfortable answering, however, it will be most helpful if you answer all questions. Your answers will be shared ONLY with the researchers for this project.

Your gender: Female Male

Year of Birth:

OPTIONAL

Cultural/ethnic background(check all that apply):

First Nations Metis Inuit

Visible minority

Immigrant

Current city of residence:

Current province of residence:

3.2 Please indicate your education qualification (Canada/Outside Canada?)

EDUCATION LEVEL YEAR OBTAINED(give your best guess)

Less than a High School Graduation Certificate

31

High School Graduation Certificate

Trade or Technical Certificate

College Diploma

EDUCATION LEVEL (Cont.) YEAR OBTAINEDgive your best

guess)

University Bachelor degree (name of degree + major)

University graduate degree (name of degree + specialization)


3.3 Please indicate your Employment History

Please provide a general idea of the kind of work you have done in the last 5 years (e.g. retail, construction, manufacturing, forestry, etc.)

How many different jobs (give your best guess) have you had in the last 5 years?

If you were unemployed in the last 5 years, please estimate the number of months you were unemployed during the last 5 years (give your best guess).

Please indicate the type of work you are hoping to find.

3.4 Please check ONE of the following BEST APPLIES to your current work situation.

EMPLOYMENT STATUS

Current work status:

Not currently working

Part-time work

Full-time work

32

3.4.1 Which statement BEST applies to you?

I want to be employed and I am looking for work

I want to pursue a training/ education program to improve my qualifications

I am not employed and I am not looking for work because:

I am a student

I am retired

I am a stay-at-home parent

Other (please specify)

I am temporarily laid off but am expecting to be called back

I am underemployed (I want to be working more hours at the same type of job)

I am underemployed (I am qualified to do more skilled, better paid work)

3.4.2 Which statement BEST applies to you?

I have never been employed

I have had some jobs for short periods (weeks or months) at a time

I have had fairly steady employment in the past

If you are currently employed, do you consider your job to be:

Not at all

Not much

A little

Quite a lot

A lot

consistent with your skills and qualifications

consistent with your vision of your preferred employment

If you are currently employed, so you consider your salary to be:

Not at all

Not much

A little

Quite a lot

A lot

consistent with your skills and qualifications

adequate for your cost of living needs

3.4.3 Other relevant factors

33

Are you a citizen or permanent resident of Canada? (i.e., legally entitled to work in Canada)

Yes No

Are you proficient in speaking either English or French?

Yes No

Are you proficient in writing either English or French

Yes No

Do you have a health problem that impacts your employability?

Yes No

Do you have access to the support systems you need (family, friends, financial)?

Yes No

Are you a parent?

No Yes, with reliable child care

Yes, with child care that is not reliable

Do you consider yourself to be a person with a disability?

Yes No

PROCESS:

INTAKE ASSESSMENT:

An Intake Interview is required for each client in the research project. If your client is an existing client, an intake interview is still required. Your assessment for an existing client will be based on their needs at the time they enter the research, not the time they entered your caseload. The research project tracks changes in the client for the six weeks of the research so your assessment of their needs upon entry into the research is critical.

4. IDENTIFICATION OF EMPLOYABILITY NEEDS: Client employability needs are organized into five broad employability dimensions: Pre-Employability/Job Readiness; Career Decision Making; Work Search; Skill Enhancement and Employment Maintenance. Each employability dimension lists a number of indicators which suggest specific needs within each dimension. No list of indicators can be complete; please use the Other category and describe any additional indicators you identify. As you record the results of your Assessment Interview with your client, focus on the indicators of strongest need.

Note that the system will default to Not At All for any indicators that you do not check off.

34

4.1 Pre-Employability/Job Readiness

Client needs assistance to : Not at all

Not much

A little

Quite a lot

A lot

Identify and clarify future direction (e.g.,training, education, employment or change in life circumstances goal)

Identify personal strengths/resources that support future direction (e.g.,training, education, employment or change in life circumstances goal)

Resolve specific challenges/vulnerabilities that may impact on future direction (e.g., mortgage, public transit, day care etc.)

Access community resources/supports that address specific personal challenges/vulnerabilities and assist in helping move goal forward (e.g., mental health services; addictions counselling; public housing etc.)

Develop necessary personal supports needed to move toward future direction (e.g., family, childcare, transportation)

Acquire life/employment skills and attitudes that support future direction

Acquire attitudes which support future direction

Develop and follow a plan of action to move future direction forward

Keep appointments and sustain efforts

Other: (please specify)________________________________

35

4.2 Career decision making


Not much

A little

Quite a lot

A lot

Identify own strengths, skills, interests, values, transferable skills

Connect strengths, skills, interests, values, transferable skills to career choices

Research work opportunities using a range of sources (e.g., job boards, labour market information, internet, networks, employer and employee contacts)

Identify a career goal (e.g., employment/training/ education/change in life circumstance)

Research details specific to career goal (e.g., time needed in education/training; future employment prospects; types of work; places of work; local opportunities)

Identify personal resources that support achievement of career goal (e.g., support system, finances, motivation)

Identify challenges, internal or external, that may interfere with achievement of career goal (e.g., mobility, local opportunities, finances, health)

Develop and follow a plan of action to mitigate challenges and move toward career goal


Other: (please specify)__________________________________

36

4.3 Work search


Not much

A little

Quite a lot

A lot

Confirm employment goal

Confirm that qualifications and experience are consistent with employment goal

Identify personal strengths that support successful work search

Resolve obstacles that may interfere with successful work search

Access and make use of resources to address obstacles to successful work search

Identify potential employers and potential employment opportunities

Adjust/adapt employment goal with employment opportunities as needed

Identify transferable skills

Complete appropriate resume and cover letter

Use networks to identify employment leads

Use resources, tools and methods to support work search including internet

Tailor resume and cover letter according to work possibilities

Demonstrate appropriate job interview skills

Develop and follow a plan of action for active work search

Demonstrate work attitudes and behaviours to support successful job search

Be active and persistent in work search

Keep appointments and sustain work search efforts

Other: (please specify)___________________________________

37

4.4 Skill enhancement:


Not much

A little

Quite a lot

A lot

Confirm training/education goal

Research future employment prospects related to training/education goal/program prior to pursuing education/training

Research training/education options available to achieve training/education goal

Evaluate and choose training/education options taking into consideration personal circumstances (eg.,supports, strengths and limitations)

Identify potential challenges that may impact on achievement of training/education goal

Proactively develop strategies to address identified challenges

Acquire study and personal skills needed to be successful in education/training

Identify resources that are available in training and education sites and/or community agencies to provide help and guidance supporting successful completion of program

Sustain motivation to complete training/education program

Other: (please specify)___________________________________

38

4.5 Employment maintenance


Not much

A little

Quite a lot

A lot

Identify important skills and attitudes that improve chances of keeping employment

Identify personal strengths and limitations with respect to these skills and attitudes

Develop a learning plan to acquire/ strengthen skills and attitudes before on the job problems arise

Identify community and/or workplace resources that provide help and guidance related to keeping work

Know job roles, responsibilities and expectations that support being successful on the job (e.g., who to report to; who makes decision; approval processes, getting answers to job related questions)

Assertively seek assistance when needed

Develop and follow a plan to remain up to date with on the job changes in duties and competencies

Sustain efforts over time

Other: (please specify)_________________________________

39

4.6 Additional Life Circumstances:

Many circumstances beyond those already identified may impact on the client’s employability outcomes. A number of possible circumstances have been listed below as well as an “Other” category.

Please check off or add additional challenges in your client’s life circumstances that may impact on client progress and success:

Indicators of challenges in client life circumstances are in evidence:

Not at all

Not much

A little

Quite a lot

A lot

Improved housing is needed

Improved transportation is needed

Improved capacity to work and/or study is needed

Increase in sense of responsibility for own choices and behaviours is needed

Increase in ability to set short and long term goals is needed

Increase in understanding expectations and demands of employers is needed

Reduction in destructive behavior is needed

Following medical and medication protocols is needed

Increased openness to change is needed




Other: _________________________________

40

4.7 Assessment Interview Results:

Your record of your Assessment Interview with your client indicated client needs in the Employability Dimensions checked off below:

4.7.1 Assessment Interview Results:

Pre-Employability

Career Decision-Making

Work Search

Skill Enhancement

Work Maintenance


Please indicate below the employability dimension(s) you and your client are currently working on:

4.7.2 Focus of Client/Practitioner Work:

Pre-Employability


Work Search

Skill Enhancement

Work Maintenance


41

4.8 Employment Goal and Action Plan:

Arriving at an agreed upon goal and a concrete action plan are fundamental. They are the roadmap to guide and to gauge progress towards the client achieving his/her employability goal. The Action Plan must be updated with each client contact. You are encouraged to complete the action plan on screen with the client, and to print an updated copy for the client each time you work together (click Print bullet).

Goal

Action steps Do this by

Finished(check)

1.

2.

3.

4.

5.



42

5. IDENTIFICATION OF PERSONAL ATTRIBUTE NEEDSPersonal attribute factors, unique for each client, are known to impact on client employability outcomes. Following your Assessment Interview, you are asked to assess five specific personal attributes for each client: Self-Management; Self-Esteem; Well-Being; Self-Efficacy; and Self Awareness. You may find it helpful to click on the More info bullet to review a number of indicators which suggest specific needs within each personal attribute. No list of indicators can be complete; please use the Other category and describe any additional indicators you identify.

As you continue to work with your clients and get to know them better, your understanding of their personal characteristics and needs will deepen. First impressions may change significantly with time. It is important to capture these changes under Progress Indictors.

Client needs assistance in the following areas: Not at all

Not much

A little

Quite a lot

A lot

Ability to Self-Manage:

More info

Developing stronger Self-Esteem:

More info

Improving sense of Well-Being: i.e.:

More info

Developing stronger Self-Efficacy: i.e.:

More info

Increasing Self-Awareness:

More info

Other (Please specify): ________________________________________

43

PERSONAL ATTRIBUTES: DEFINITIONS

SAMPLE INDICATORS

more info: Self-Management:

“skills and strategies by which individuals direct their own activities towards achieving objectives; includes goal setting, decision making, planning, scheduling, staying on track”

Follows steps in action plan

Uses community supports

Manages responses to challenging situations

Presents self well

Plans and prioritizes

Takes action steps independently

Attends sessions as agreed to

Makers fewer excuses

Follows through on commitments

Takes ownership

Self-directs (pursuing what they are interested in rather than what others think they should do)

Is solution focused

more info: Self-Esteem:

“a person’s overall emotional evaluation of his or her own worth, an attitude to self and a judgement of oneself. Self-esteem encompasses beliefs (e.g., I am competent; I am worthy) and emotions (pride, shame). Self-esteem is the positive or negative evaluation of the self”

Demonstrates an optimistic outlook

Sees self as competent and able

Has confidence in ability to interact with others

Uses positive self-talk

Acts with little hesitation

Raises few objections

Asks questions

Articulates skills assertively

Makes eye contact

Is hopeful

Takes reasonable risks

more info: Well Being:

“a person's quality of life”. This is influenced by a range of factors, including work, family, community, health, personal values, personal freedom, and a person's financial situation.

Shows drive in pursuing own goals

Is open to suggestions to support action plan

Is establishing/using own support system

Manages his/her personal situation adequately

Involved with family, child-rearing and community

Maintains a healthy lifestyle

Makes better choices than before

44

Networks with all community supports

more info: Self Efficacy:

“belief in one’s capabilities to organize and execute the courses of action required to manage prospective situations; a person’s belief in his or her ability to succeed in a particular situation”:

Expresses confidence that a positive outcome is possible

Believes in own capacity to achieve goal despite obstacles

Takes steps independently

Views challenging problems as tasks to be mastered

Takes strong interest in activities they participate in

Recovers quickly from setbacks

Understands choice and consequence (If I do this, then……)

Is open-minded

Expresses self well

Uses “me” and “I “ language

Recognizes own accomplishments (I did it!)

more info: Self-Awareness:

“having a clear perception of your personality including strengths, weaknesses, thoughts, beliefs, motivations and emotions”

Understand own strengths and challenges

Sets employability goals that are achievable


Knows own personal competencies

Knows how to interact with others

Understands own personal challenges

More focused onresults and own ability to achieve them

More articulate when talking about self

Recognizes the need for other professional help

Expresses self with confidence

More realistic and more specific


45

6. WORKING ALLIANCE AND CLIENT ENGAGEMENT: The quality of the working alliance between client and career practitioner and the degree to which the client is actively engaged in the process are known to impact on client employability outcomes. The Working Alliance is the partnership developed between you and your client in order to assist the client to achieve his/her employability goal. You are asked to assess the Working Alliance and the degree of client engagement in working with you.

Again these may change as you continue to work together and it will be important to record these changes under Progress Indicators.

6.1 Working Alliance

The client and I: Not at all

Not much

A little

Quite a lot

A lot

established a climate of trust and comfort in working together

arrived at a goal that is owned by the client

agreed on the action plan steps to help achieve the client’s goal

6.2 Client Engagement:

The client: Not at all

Not much

A little

Quite a lot

A lot

participated actively in the interview

was focused on achieving results

46

47

7. PRACTITIONER INTERVENTION RECORD

U NAME IT: NEW INTERVENTIONManagement has provided us with a list of services provided by each office participating in the research. The list below is the service menu for your office. You are asked to indicate what kind of service your client received after each client contact. An “Other” category has been included.

48

7.1 Action Plan:

Each time you enter a client number, you will see a listing of all interventions done to date with that client. You can readily access the Assessment Interview data, Intervention information and comments and all of the developed Action Plans.

The Action Plan must be updated after each client contact. You are encouraged to review the previous action plan with the client as well as update the action plan on the screen with the client, print an updated copy for the client and provide it to your client so he/she can readily track their own progress. (click Print bullet).

Goal

Action steps Do this by

Finished(check)

1.

2.

3.

4.

5.



49

8. PROGRESS/CHANGE INDICATORS: It is important to record any indicators of client progress/change which are evident and which you think are noteworthy after each service contact with the client. These indicators may be actual client statements of change or they may be your impressions. They may be changes in movement forward or they may be setbacks.

You can readily retrieve the results of your Assessment Interview so you can see at a glance where you assessed your client to be and determine if any significant progress/change has occurred that should be noted. You can also add any additional needs may emerge as you continue to work with your client. You can readily access any Progress/Change report you have entered.

Progress/change takes time and there may not be significant change between contacts. If no indicators of client progress are evident and noteworthy after an individual service contact, there is no need to record.

50

8.1 PROGRESS/CHANGE INDICATORS: EMPLOYABILITY DIMENSIONS

Pre-Employability/Job Readiness


Not much

A little

Quite a lot

A lot

Identify and clarify future direction (e.g.,training, education, employment or change in life circumstances goal)

Identify personal strengths/resources that support future direction (e.g.,training, education, employment or change in life circumstances goal)

Resolve specific financial challenges/vulnerabilities that may impact on future direction (e.g., mortgage, public transit, day care etc.)

Access community resources/supports that address specific personal challenges/vulnerabilities and assist in helping move goal forward (e.g., mental health services; addictions counselling; public housing etc.)

Develop necessary personal supports needed to move toward future direction (e.g., family, childcare, transportation)

Acquire life/employment skills that support future direction

Acquire attitudes which support future direction



Other: _________________________________

51

8.2 PROGRESS/CHANGE INDICATORS:

Career decision making


Not much

A little

Quite a lot

A lot



Research work opportunities using a range of sources (e.g., job boards, labour market information, internet, networks, employer and employee contacts)

Research details specific to career goal (e.g., time needed in education/training; future employment prospects; types of work; places of work; local opportunities)

Identify a career goal (e.g., employment/training/ education/change in life circumstance)

Identify personal resources that support achievement of career goal (e.g., support system, finances, motivation)

Identify challenges, internal or external, that may interfere with achievement of career goal (e.g., mobility, local opportunities, finances, health)



Other: _________________________________

52


Work search


Not much

A little

Quite a lot

A lot













Demonstrate work attitudes and behaviours to support successful job search



Other: _________________________________

53


Skill enhancement:


Not much

A little

Quite a lot

A lot




Evaluate and choose training/education options taking into consideration personal circumstances (eg.,supports, strengths and limitations)






Other: _________________________________

54


Employment maintenance


Not much

A little

Quite a lot

A lot









Other: _________________________________

55

8.6 Additional Life Circumstances:

Please check off or add additional challenges in your client’s life circumstances which are identified over the course of working with your client. Please also indicate if there have been changes in movement forward or setbacks in any challenges identified in the Assessment and/or earlier interviews.


Not at all

Not much

A little

Quite a lot

A lot













Other: _________________________________

56

PROGRESS/CHANGE INDICATORS: 9. PERSONAL ATTRIBUTES ASSESSMENTAs you continue to work with your clients and get to know them better, your understanding of their personal characteristics and needs will deepen. The quality of the Working Alliance may also change. It is important to capture these changes as Progress Indicators, remembering that Progress means Change, either positive or negative.


Not much

A little

Quite a lot

A lot


More info


More info


More info


More info


More info


57


SAMPLE INDICATORS






Presents self well






Takes ownership


Is solution focused









Asks questions


Makes eye contact

Is hopeful











58











Is open-minded

Expresses self well











More focused on results and own ability to achieve them






59

10. WORKING ALLIANCE AND CLIENT ENGAGEMENT: You will find it useful to reflect after each interview on the quality of the working alliance you have established with your client and their level of involvement in the process. Please indicate if there have been changes.

10.1 Working Alliance:


Not much

A little

Quite a lot

A lot






Not much

A little

Quite a lot

A lot



60

11. OUTCOMES

U NAME IT: OUTCOME INDICATORS

EXIT FROM THE RESEARCH INTERVIEWOutcomes are to be recorded at the end of the 6 week research period for each client or when the client exits from services if this happens before the 6 week research period. Exit from the research does not mean the end of your work with the client. You will continue to work with clients as needed but using your own data collection system.

For the research project, your assessment of the amount of change your client has experienced in the time you worked together or the 6 weeks, whichever comes first, is essential. Following the Assessment Interview, you completed checklists indicating where you thought your client was at that time. You have noted and recorded Progress Indicators after each interview and/or each service your client received during the 6 week period. You are asked to indicate where you think your client is NOW at the end of the research period. Please indicate all areas where you think client change has occurred, positive or negative. If you think no change has occurred and the client has not progressed or gone backwards, you will leave your ratings the same.

61

Changes in Client Employability Dimensions:

11.1 Pre-employment:

Client continues to need assistance to: Not at all

Not much

A little

Quite a lot

A lot

Identify and clarify future direction (training, education or employment goal)

Identify personal strengths/resources that support future direction (training, education or employment goal)

Resolve specific challenges/vulnerabilities that may impact on future direction

Develop necessary personal supports to move toward future direction (eg. family, childcare, transportation)

Access community resources/supports that address specific challenges/vulnerabilities and assist in helping move goal forward

Acquire life skills and employment skills and attitudes that support future direction





62

11.2 Career Decision Making


Not much

A little

Quite a lot

A lot



Research work opportunities using a range of sources (job boards, labour market information, internet, networks, employer and employee contacts)

Identify a career goal (employment/training/education)

Research details specific to career goal (time needed in education/training; future employment prospects; types of work; places of work; local opportunities)

Identify personal resources that support achievement of career goal (eg. support system, finances, motivation)

Identify challenges, internal or external, that may interfere with achievement of career goal (eg. mobility, local opportunities, finances, health)





63

11.3 Work Search


Not much

A little

Quite a lot

A lot




Resolve obstacles that may interfere with successful work search

Access and make use of resources to address/ mitigate obstacles to successful work search














64

11.4 Skills Enhancement


Not much

A little

Quite a lot

A lot




Evaluate and choose training/education options taking into consideration personal circumstances (supports, strengths and limitations)








65

11.5 Employment Maintenance


Not much

A little

Quite a lot

A lot











66

11.6 Changes in Life Circumstances:

You are asked to indicate where you think your client is NOW at the end of the research period. Please indicate all areas where you think client change has occurred, positive or negative. If you think no change has occurred and the client has not progressed or gone backwards, you will leave your ratings the same.


Not at all

Not much

A little

Quite a lot

A lot















67

12. EMPLOYABILITY DIMENSION(S) FOCUSPlease indicate the employability dimension(s) which was/were the focus of your work with your client during the research period:

Pre-Employability


Work Search

Skill Enhancement

Work Maintenance


68

13. PERSONAL ATTRIBUTES ASSESSMENTYou are asked to indicate where you think your client is NOW at the end of the research period. Please indicate all areas where you think client change has occurred, positive or negative. If you think no change has occurred and the client has not progressed or gone backwards, you will leave your ratings the same.


Not much

A little

Quite a lot

A lot


More info


More info


More info


More info


More info


69


SAMPLE INDICATORS






Presents self well






Takes ownership


Is solution focused









Asks questions


Makes eye contact

Is hopeful











70











Is open-minded

Expresses self well











More focused on results and own ability to achieve them






71

14. WORKING ALLIANCE AND CLIENT ENGAGEMENTPlease give your overall assessment of the quality of the working alliance you established with your client over the time you worked together. Please also indicate your overall assessment of the involvement of your client in the work you accomplished together.

14.1 Working Alliance


Not much

A little

Quite a lot

A lot






Not much

A little

Quite a lot

A lot



72

15. LABOUR MARKET OUTCOMES: You are asked to indicate where your client is NOW at the end of the research period with respect to his/her employment and/or training/education.

EMPLOYMENT STATUS

employed in training/education program or waitlisted

neither employed nor in training

15.1 Employed:

Type:

Full-time

Part-time

Contract

Location:

Within 50 km of residence

Home only on weekends

Home only periodically

15.2 Fit: Not at all

Not much

A little

Quite a lot

A lot

Work is consistent with client skill and qualification levels

Work is consistent with client vision of preferred employment

15.3 Salary Not at all

Not much

A little

Quite a lot

A lot

Salary is consistent with client skills and qualification

Salary is adequate for client cost of living needs

73

15.4 In Training/Education Program or waitlisted:

Enrolled in Training/Education Program:

Full-time

Part-time

Waitlisted for Training/Education Program:

Full-time

Part-time

Duration of Training/Education Program:

2 weeks or less

3 weeks to 3 months

4 months to 6 months

More than 6 months

Location of Training/Education Program:

Within 50 km of residence

Home only on weekends

Home only periodically

15.5 Fit of Training /Education Program

Not at all

Not much

A little

Quite a lot

A lot

Training/Education program is highly consistent with client employment goal

Training/Education program is well linked to employment opportunities locally and regionally

74

15.6 Neither employed nor in Training/Education:

The client wants to be employed and is I am looking for work

The client wants to pursue a training/ education program to improve his/her qualifications

The client is not employed and is not looking for work because he/she is:

a student

retired

stay-at-home parent

Other (please specify)_______________________________________

The client is temporarily laid off but am expecting to be called back

The client is underemployed (wants to be working more hours at the same type of job)

The client is underemployed (qualifies to do more skilled, better paid work)

75

76

77

Protocol for Exit from the Research1. Have on hand: Final Questionnaire and sharpened pencils with erasers, Gift Card

and Thank You Letter.

2. Greet the client and express appreciation for their participation.

3. Explain that filling in the Final Questionnaire is a very important part of the study. The questionnaire asks questions about the client’s experience with the programs and services, what they have learned and what, if anything, has changed for them as a result of receiving services. There are no wrong answers and we encourage clients to be as honest as possible. This questionnaire will take about 20 minutes to complete.

4. Give the client the questionnaire and provide an explanation on how to respond to the questionnaire. Go to the Script for the Final Questionnaire– use this as an intro to and explanation of the survey. The script suggests doing 1 example – please feel free to do 2 or 3 – do it until you believe the clients are comfortable answering. Invite questions or requests for clarification. State that you will answer any questions they have on any item as they go through the survey.

5. Invite the client to complete the questionnaire on paper in your office or online. If on paper, indicate that you will answer any questions they may have. If online, send the questionnaire to the client’s e-mail immediately. Ensure that you have made arrangements for them to pick up their gift card after you have received the questionnaire back.

6. Once completed, present the clients with their gift cards and thank you letter. Reinforce how important their contribution has been to helping improve services.

7. Collect all questionnaires completed on paper and ensure they are sent weekly to CCDF.

8. If completed online, forward questionnaires as soon as received to Marie-Anne Bedard at [email protected].

78


Suggested Script-: Final Questionnaire

(We suggest that you talk through an example for clients before they begin)

There is one questionnaire the researchers need you to complete. The scale on this survey is different and I would like to walk you through a couple of examples to be sure the instructions are clear before you begin.

This questionnaire asks you your opinion on a five point scale and you are asked to respond in two steps – the first to decide whether each statement is adequate (OK) or not adequate (Not OK) and then to decide how OK or how Not OK using the five point scale.

What you are asked to do for this survey is to remember back to the beginning of the study and knowing what you now know, to rate what you knew or felt before this study started and then to rate what you know or feel now at the end of the study.

You may also find it useful to use an everyday example. Let’s say you have a perfect driving record but you have never taken a course in driving in icy conditions – you just always do it well. However you decide to take the course just out of interest and you discover a lot of helpful tips you never knew before. If you had been asked before the course if you knew about how to drive in icy conditions, you likely would have given yourself a very high rating---after all, you have been accident free for many years. After the course however, you think back to what you thought you knew before the course and realize that you did not know all there was to know. Knowing what you know after the course, you might have given yourself only a medium rating before compared with what you know now. Of course, sometimes you are as good as you think you are and there may be no difference. Thinking about before and after a learning experience helps bring out some useful information. This survey asks you to think about where you were at the beginning of participating in this study and where you are now.

Let’s take an example from the survey itself:

The first item says:

“A clear understanding of what I need to do to move forward in my career.” Knowing what you know now about having a clear understanding of what you need to move forward, where were you at the start of the study?

79

Step 1: Were you Not OK or OK with respect to having a clear understanding? (Invite the client to answer).

Step 2: If you were not OK, use the scale to decide how not OK (0 indicating not adequate and 1 indicating almost OK).

If you were OK, use the scale to decide how OK (2 means just barely OK, 4 is exceptional or 3 means somewhere between 2 and 4).

Have the client(s) enter their response in the Before columns.

Step 3:

Now repeat the same process but rate yourself at the end of the study.

“A clear understanding of what I need to do to move forward in my career.” How would you rate yourself now with respect to having a clear understanding? Again:

Step 1: Are you now Not OK or OK with respect to having a clear understanding? (Invite the client to answer).

Step 2: If you are not OK, use the scale to decide how not OK (0 indicating not adequate and 1 indicating almost OK).

If you are OK, use the scale to decide how OK (2 means just barely OK, 4 is exceptional or 3 means somewhere between 2 and 4.

Have the client(s) enter their response in the After columns.

One example may be enough for the clients to get the idea correctly. Of course, provide additional coaching if and as needed.

80

Before Workshop

This is an excerpt from a questionnaire used in an earlier research project using the same scale we will use in the questionnaire in this project. The actual questionnaire will be sent out to you in the coming weeks. This excerpt is included so you can become comfortable with the scale and how to explain it to your clients and have any questions answered in the Orientation Session. This is a SCALE EXAMPLE ONLY.

First Some General Questions About Your Career Planning You agreed to participate in a Research Study about four weeks ago. We would like to know what has happened over these four weeks. Below are several statements. For each statement, we are asking you to do two things. Keeping in mind what you know now about using labour market information for career decision making or job search, please think back to four weeks ago and indicate in the BEFORE column how OK you were with respect to this statement at that time. Next, think of NOW and in the AFTER column, indicate how OK you are now with respect to the statement.

To help you provide a more accurate answer, please use the two-step decision-making process described below when responding.

(A) decide on whether the statement was/is adequate (OK) or not adequate (Not OK), then

(B) assign the appropriate rating:

(0) not adequate,

(1) not really adequate, but almost OK,

(2) adequate, but just barely (still OK otherwise it would be 0 or 1),

(4) exceptional,

(3) somewhere between minimally OK and exceptional.

Graphically, the scale looks like this:

Knowing what you know now about using labour market information for career decision making or job search, rate

Before After

81

Not OK OK

1 2 3 40

yourself before the research project and rate yourself now

I had/have

1. A clear understanding of what I need to do to move forward in my career

2. A clear vision of what I want in my career future

3. Reviewed my past work, education and experience so I know what strengths and skills I have

4. A list of possible career options that I want in my career future

5. Knowledge of print and online resources that help me to research career/employment options

6. Confidence that career-related employment opportunities actually exist that fit with what I want in my career future

7. Identified my career/employment-related goals and the next steps to get there

82

The research team for the Common Indicators: Transforming the Culture of Evaluation in Career and Employment Services study

would like to pass on our sincere appreciation to you for participating in this study.

Without your help we could not have gathered the data to understand the results of our programs and services and how to keep improving them. In helping us we hope you have benefited from the programs

and services you have received.

With appreciation:

Lynne Bezanson and Celine Renald at the Canadian Career Development Foundation in Ottawa

Dr. Dave Redekopp, President, Life-Role Development Group, Edmonton

Ken Organ, Goss-Gilroy, St. John’s, Newfoundland

The Canadian Research Working Group in Evidence Based Practice in Career Development (CRWG)

…and the manager and staff at the Employment Office

We extend a big thank you and wish you the best of luck in the future with your career journey!!

83

84

85

U NAME IT LOG

Research Practitioner:

Date:

Would you be willing to be interviewed by one of the researchers re your feedback? Yes No

1. What part(s) of U Name It are you finding to be most useful? Please be specific:

2. What part(s) of U Name It are you finding to be least useful? Please be specific:

3. Are there parts of U Name It that you are finding difficult and/or confusing to use? Please be specific and please explain what is difficult and/or confusing:

4. How long on average is it taking to enter data into U Name it each time?_____________________ minutes.Please send feedback to Lynne Bezanson at [email protected]. Thank you!

86


Client Recruitment and Follow Up Record

RESEARCH PRACTITIONER:

OFFICE:

RECRUITMENT INTO THE RESEARCHASSESSMENT INTERVIEW

PROVISION OF REGULAR SERVICES

EXIT FROM THE RESEARCH

CLIENT QUESTIONNAIRE COMPLETION AND CLIENT ACKNOWLEDGEMENT

JANUARY 21-25 JANUARY 28-MARCH 8 MARCH 11-15 MARCH 11-15

JANUARY 28-FEBRUARY 1

FEBRUARY 8-MARCH 15


FEBRUARY 4-8 FEBRUARY 11-MARCH 22


87

Documents

Project Overview - crwg-gdrc.ca€¦ · Web viewProject Overview. Common Indicators: Transforming the Culture of Evaluation in Career and Employment Services