Learning Good Employee Skills: Maximizing Internship ... · Learning Good Employee Skills: Maximizing Internship Program Effectiveness. NSEE Annual Conference, September 2016. Abby

Learning Good Employee Skills: Maximizing Internship Program

Effectiveness

NSEE Annual Conference, September 2016

Abby Trout, Career CenterCarol Trosset, Institutional Research and Assessment

Carleton College

1Institutional Research and Assessment

Overview

Carleton College = a highly selective liberal arts college in Minnesota, with 2000 undergraduates.

• Introduce the Carleton College Career Center’s internship process and its focus on learning goals

• Review Trosset’s research at Bennington College on which skills are most valued by interns’ employers

• Present new research at Carleton on how students become aware of and learn these skills

• Discuss implications for Career Center programming

Institutional Research and Assessment 2


Career Center mission:


Carleton Career Center learning goals:• Self-assessment • Career field awareness• Transferable skills • Market selves • Experience• Job search• Graduate degrees• Access to networks• Effective networking


Career Tracks


Internships at Carleton

• 488 (33%) of rising Carleton sophomores, juniors, and seniors did internships in Summer 2016

• 113 of these internships were funded through the Career Center

• Career Center provides funding to cover expenses (i.e., housing, transportation, food)

• $386K awarded for Summer 2016• Internships located all over the world (all over US and 29

countries) and in a variety of industries


What makes for a rewarding internship?

• A strong relationship with a supervisor• Learning• Goals• Self-discovery

At Carleton, students specify learning goals:

Institutional Research and Assessment

Research on Learning Outcomes


• What are the actual learning outcomes?• What are the most important/beneficial outcomes,

that internships should foster?• How can we assess (a) the degree to which students

achieve these outcomes, and (b) the quality of their performance as interns?

• Can we identify how beneficial learning took place, so we can design experiences that maximize these effects for other students?

Bennington College Research: Assessing Internship Performance

Liberal arts college in southwest Vermont700 undergraduates7-week internship required every winterAll students complete 4 internshipsResearch in 2013-14 and 2014-15Holly McCormack, Dean of Field Work Term


What are internships for?

•Students want to explore possible career paths, and to start building a resume that will lead to jobs after college

• Chronicle of Higher Education and NPR’s Marketplace 2012 study “The Role of Higher Education in Career Development: Employer Perceptions” found that

–Employers place more weight on experience, particularly internships and employment during school vs. academic credentials including GPA and college major when evaluating a recent graduate for employment.

–Among all industry segments, an internship is the single most important credential for recent college graduates to have on their resume.

–60% of employers practice intern-to-permanent hiring.


Liberal Arts Learning vs. Job Training

•AACU 2013 study “It Takes More than a Major: Employer Priorities for College Learning and Student Success” found that employers endorsed a blended model of liberal and applied learning—practices that involve students in active, effortful work.

• Bennington Field Work Term’s goal: Students will apply their academic learning in a broader context outside the classroom.

• Same AACU study asked employers whether they want employees to have the skills liberal arts colleges already value, like writing and critical thinking. Employers said “yes.”


But what do interns need to learn?

Supervisors of Bennington students wrote open-ended evaluations, listing interns’ strengths and areas needing improvement.What did they write about without being prompted?• “She was always on time and willing to step in and help with anything.”• “He asked great questions and responded to feedback gracefully.”• “She could pay more attention to her body language during meetings and

while working. At times she seemed a bit distracted and disinterested.” • “He tended to drift off task to check Facebook.”


Categories employers mentioned:


• Work ethic• Engagement with work• Quality of work• Organization/efficiency• Punctuality • Professional conduct• Takes direction• Teamwork

• Interpersonal skills• Independence/initiative• Learns quickly• Confidence• Creativity• Writing skills• Critical thinking skills• Other job skills

Note: Most of these are neither liberal arts skills nor job-specific skills. Instead, they are qualities of any good employee in any job sector.

Qualities of a good student?(taken from Bennington Faculty narrative evaluations)

• Work ethic – “works hard,” “diligent,” “spends time texting during class”• Engagement – “excellent work when focused but not consistently engaged”• Quality – “does high quality work,” “should focus on producing high-quality work

the first time around”• Organization – “sometimes unprepared,” “talented but disorganized”• Punctuality – “late to class several times,” “some absences”• Takes direction – “responded to feedback,” “needs to learn to follow directions”• Teamwork – “listens and contributes well in discussion,” “good team member”• Initiative – “challenges himself,” “does the minimum to get by”• Learns quickly – “learns from mistakes,” “learns quickly”• Confidence – “should have more confidence,” “wish would speak up more”


Evaluating internship performance


Unacceptable Needs Improvement

Good Exceptional

Focused work ethic

Too laid-back, easily distracted, takes frequent breaks

Mostly works hard, but some

tendency to lose focus and drift off

task

Productive and conscientious

Works harder than others and is much

more productive

Engagement with the work

Seems unmotivated, avoids or resists unwanted tasks

Engagement and motivation vary

with the task

Engaged in the work, willing to help as needed

Seizes every opportunity to

learn as much as possible

Organization and Efficiency

Inefficient, forgets about assignments,

may not follow through to completion

Still learning how to manage time

and priorities when working

alone

Good time management, follows up on assigned tasks

Unusually efficient, intuitive sense of priorities

Takes direction Fails to incorporate feedback into work

products; rarely checks in with

supervisor

Some tendency to put own agenda ahead of others’

needs

Checks in regularly, responds

appropriately to feedback

Frequently seeks out feedback and uses it to improve

the work

16

Benefits of the rubric:• All employers are asked to comment on a consistent group of skills • Students have a consistent definition of what the College means by

workplace readiness• Students can:

–compare their self-rating using the same rubric to that of their employers

–track their rubric ratings over four years–use employer evaluations as part of a job application packet

• Employers have asked us for copies of the rubric to use with their staff


Challenges for using rubrics with employers:

•Colleges need to compare the students in aggregate, and over time.

•But students all have different jobs and employers.•The students start at different stages of development.•The employers have different expectations.•We cannot norm the employers to consistent standards.•The evaluation process cannot be blind or unbiased.


Carleton College Research:Employee Skills Metacognition

Focus was on tracking and increasing student awareness of these skills.

90 of the funded interns in Summer 2016 allowed us to use their materials for this research.

Materials included:Application essays Blog postsLearning contracts Reflective essays

Rachel Leatham, Program Director


20

Student Motives and Preparation

Motives

Support the mission – 44%Interest in activity – 39%Personal connection – 19%Learn skills – 17%Explore a career – 13%Apply academics – 9%Broaden my horizons – 6%

Preparation

Coursework – 52%Previous “job” – 39%Specific skills – 33%Studying the organization – 17%Off-campus study – 10%Local knowledge – 10%Extracurricular activity – 9%


Students’ Learning Goals• Explore a possible career – 73%• Gain academic learning – 47%• Learn job-specific skills – 41%• Improve research skills – 22%• Improve communication skills – 13%• Improve time management skills – 13%• Improve foreign language skills – 12%• Gain cross-cultural competency – 11%• Improve interpersonal interaction skills – 11%• Gain life skills – 7%

32% had only job-specific or academic goals.


Learning Outcomes from Final Reflections


Goal Outcome Goal Outcome

Work ethic 0 3 Takes direction 1 12

Engagement 0 6 Teamwork 8 9

Quality of work 0 9 Interpersonal skills 1 7

Time management 12 10 Initiative 0 4

Flexibility 1 6 Confidence 0 6

Professionalism 3 3 Communication skills 12 13

What They Learned about Employee Skills

Work Ethic• The importance of persistence• How much work is required to develop a good product• How to time breaks to be more productive

Engagement• That feeling engaged with one’s work improves its quality• That personal growth happens when you commit yourself to

something


What They Learned, cont.

Quality of Work• The importance of doing high-quality work• Attention to detail• The value of being well-prepared

Time Management• Improved time management skills• Became better organized• The importance of managing one’s time• How to set priorities



Flexibility• The importance of flexibility• The value of patience• That being flexible can improve the quality of the work

Taking Direction• The importance of asking questions• The importance of taking notes• How to accept critique



Teamwork• The importance of teamwork• The importance of learning from co-workers• Teamwork and facilitation skills• Project management and leadership experience

Interpersonal Skills• The value of interacting and building relationships with

co-workers


Those with a goal more likely to learn;Most who learned did not have that goal.


0%

10%

20%

30%

40%

50%

60%

Learned target skill Learned different skill No employee skills reported

With Goal

No Goal

What leads to the learning when it’s not motivated by a goal?

• Specific on-the-job events• The difference between a job and a class• Directed reflection


Specific events• “One of the biggest lessons I learned early in my internship was to not be afraid to

ask for help, or be inexperienced in any particular skill. The first project I was assigned involved using Excel. Initially I was frustrated and embarrassed that it took me longer than expected to learn how to use that program and that I didn’t know how to use it before. However, as I reflected with my supervisor about this anecdote at the end of my internship, she was surprised that I didn’t ask her for help rather than dealing with it on my own. At the time I didn’t know that was an option, I believed the expectation was to already be equipped to complete the projects assigned.”

• “Staying organized was something that I struggled with sometimes. My supervisor would send me an e-mail of something that would be coming up in two weeks and because it was two weeks away I’d say, ‘Well, I’ll read it next week when I have more time.’ I would then lose my e-mail in my constantly overflowing inbox and I’d have to ask my supervisor to resend it. Early on, at meetings I wouldn’t take notes, thinking my memory would be good enough, but when it became clear that the little nuances and wordings mattered just as much as the main idea we were trying to convey I realized that I had to start taking some notes or become lost.”


Events, cont.

• “My supervisor was an extremely busy woman – as a result she usually sent tasks to me and my two fellow interns and left it up to us to work together or divvy up the work as we thought best. This was an important exercise in teamwork: I learned the importance of giving credit where credit was due was essential for a successful team. Even small things like saying ‘Ashley and I edited this report’ rather than ‘here’s the report you wanted edited’ goes a long way in creating a healthy group dynamic.”

• “I’ve learned to keep my collected data clean. Although this project had been going on for three years now, there was no standardized manner of storing or analyzing the data. Thus, the last two weeks of my internship were largely spent going through thousands of haphazardly sorted photos, essentially doing other interns’ work for a second time. This was incredibly frustrating. Acknowledging that I have been lax in storing my data neatly during past projects, I learned the hard way that it pays to stay organized from the beginning!”


Difference between a job and a class

• “Through this project I developed my willingness to pay attention to detail as I encountered many specific comments of constructive criticism, a type of conversation I rarely have in an academic setting as little of my work ever reaches beyond my professor grading a specific assignment. This experience helped me understand the kind of work put into public education initiatives and the importance of detail-oriented review.”

• “Usually I was very engaged in my work because I had to be extremely attentive when processing information and working on my projects. In the field that I am interning for, mistakes must be recognized and fixed before it is sent in.”

• “Here, giving up means letting over 40 children down. Giving up means sending the message to those children that they aren’t worth it. Giving up meant impacting someone else’s life just as much as my own. In school, giving up in a class, for example, only really affects me. I guess in other words I learned a greater sense of responsibility, because I have wanted to just quit so many times.”


Directed Reflection via Weekly Blog Prompts

• Observe others’ work ethic• How you stay engaged with the work• What you’ve learned by making a mistake• Standards of punctuality• Observed nature of professional conduct• Difference between a supervisor and a professor• How people work in teams• Relevance of interpersonal skills• Working independently and taking initiative• Accomplishments and skill building• Observe effective communication


Among those who reflected on learning non-goal employee skills, 3/4 wrote about employee skills in response to summer blog prompts.

• “The skill that I’m most proud of is my growing sense of flexibility. I tend to be a rather stubborn person, and I don’t like to change things or let on when I am lost. This internship, however, has really challenged me to have to change this. Every time I think that I am finally done with creating my stimuli, my supervisor thinks of something I can change. My original plans and expectations for my experiments have certainly changed a lot over the course of the past five weeks, and I think that’s absolutely awesome. Though it’s certainly frustrating to feel as if I’m not making progress at times, it’s really cool seeing just how much of a process experiment design is.”

• “Even doing boring tasks (sharpening pencils, printing tickets, folding programs), I am surrounded by friendly and engaging people. Many of my bosses are on teams that put on shows here, so seeing their shows serves as motivation and a reminder that grunt work is almost always necessary.”


Supervisor Evaluations


Positive Improvement Needs Work

Work ethic 22 2 0

Engagement 10 0 1

Quality work 9 2 1

Time management 6 2 1

Flexibility 5 3 0

Professionalism 3 0 1

Take direction 8 2 2

Teamwork 6 7 2

Interpersonal 15 3 4

Initiative 11 5 2

Confidence 2 12 4

Communication 5 10 0

Differential Salience

• Students start out knowing the importance of time management and communication skills.

• They become aware of the importance of asking questions, attention to detail, and relationships with co-workers.

• They remain unaware of most dimensions of professionalism (other than dress code).


Next Steps•Conduct focus groups with some of the 2016 interns about

what facilitated this learning, and how to get other students to see its importance.

• Summer Interns 2017–Require one of the three learning goals to be a generalizable skill.– Continue to adjust blog prompts to include features of employee skills we know students are encountering, and that they should think about.

– Find ways to expand their awareness of the less salient skills (like professionalism).

–Use metrics to show incentive to other interns to register with our office


Next Steps

•Broader application of specific findings and the usage of structured reflection pre-, during, and post-experience to other preparation programs

–Post-grad fellows –On-campus jobs and interns–Externships

•Structure alumni-student programming–Build alumni profile questions around employee skills prompts–30 Minutes program questions


Questions?

Documents

Learning Good Employee Skills: Maximizing Internship ... · Learning Good Employee Skills: Maximizing Internship Program Effectiveness. NSEE Annual Conference, September 2016. Abby