2

What’s Happening to Our “Freshout” Engineers?

https://sites.googboisestate.edu/faculty/svillachica.htmle.com/a/boisestate.edu/ieeci/e2r2p

Steven W. Villachica Anthony W. Marker Donald Plumlee

Amy ChegashProject Worldwide Out of the Box

3

Engineering Education Research to Practice (E2R2P)Portions of this material are based upon work supported by the National Science Foundation under Grant No. 1037808.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

The Research Team

¨ Don Plumlee, PhD.¨ Steve Villachica, PhD.¨ Tony Marker , PhD.¨ Linda Huglin, PhD.¨ Drew Borreson¨ Shannon Rist¨ Amy Chegash¨ Lorece Stanton¨ Jessica Scheufler

Business Plan• Ray Svenson

4

Agenda

¨ Share our exploratory research

¨ Ask for your help interpreting the data we’ve collected

¨ Dialog, not an information dump

¨ Wrap up

5

Why Should You Care?

¨ Engineers create a lot of the stuff we use. Universities can’t mint them fast enough.

¨ 64% engineering employers are “somewhat satisfied” with quality of new hires.(Spinks, 2006; Trevelyan & Tilli, 2008; Trevelyan, 2010; Blom & Sakei, 2011)

¨ Professional skills for the engineering workplace include teamwork, communication, coordination, data analysis and problem solving.(Hoey & Gardner, 1999; Jonassen et al., 2006; Grant & Dickson, 2006; Korte, Sheppard, & Jordan, 2008; Trevelyan, 2007, 2008; Anderson et al., 2011; Passow, 2012; ASEE & NSF, 2013)

6

Research Design

What sort of engineers do engineering firms REALLY want to hire?

Boundary Crossing CompetenciesCommunication, teamwork, networks, critical thinking, global understanding,

perspective, organizational culture, project management, etc.

Many Disciplines Many Systems

De

ep a

t least o

ne

discip

line

De

ep a

t least o

ne system

(c.f. Brown, 2005; Spohrer, 2010; ASEE & NSF, 2013)

ME

7

Research DesignDecrease Ramp-up Time to Competent Job Performance in the Engineering Workplace

Research Questions

• What are newly graduated and hired “fresh out” engineers doing/not doing in the workplace that they should?

• What are the consequences of performance/non-performance in the workplace?

• What are the root causes of workplace nonperformance?

Mixed Design: Focus Groups & Surveys

• Engineering managers, engineering leads, HR personnel, and technical scientists who work with fresh out engineers

• Fresh out engineers• Professional engineering organizations

8

Actual Competency

New Task/Project

Leave University/Enter Workforce

Literature Review

Desired Competency

Promotion!

$$$

Pe

rfo

rma

nc

e

Time

Company Costs$ Training$ Errors$ Mentoring$ Salary$ Opportunity$ Other projects$ Others?

{REDUCE CO$T

• Improve Starting Skills

• Change Performance Curve

• Make Boundaries Porous }

9

Literature ReviewThere is a significant disconnect between engineering education and engineering practice.

(Bucciarelli & Kuhn, 1997; NAE, 2005; Jonassen et al., 2006; Spinks et al., 2006; Korte et al., 2008; Trevelyan, 2008, 2010; McCrohon & Gibson, 2009; Sheppard et al., 2009; Morgan & O’Gorman, 2010; Anderson et al., 2009, 2010; Duderstadt, 2010; Stump et al., 2011; ASEE & NSF, 2013; Winters et al., 2013)

10

Literature ReviewThe time for freshout engineers to fit into their jobs and perform them competently is a significant workplace cost.¨ 2 to 5 years ramp-up.

(Trevelyan & Tilli, 2008; Jonassen et al., 2006)

¨ Socialization and onboarding are long-lived. (Dai & De Meuse, 2007; Bradt & Vonnegut, 2009; Roethle, 2012; Jones, 2013)

¨ The engineering workplace supports socialization and onboarding variably well. (Montesano, 2007; Roethle , 2012; Korte & Lin, 2013)

¨ Bad onboarding is co$tly. (Ramlall, 2004; Rollag et al., 2005; Snell, 2006; Kowtha, 2008; Lindo, 2010; Roethle, 2012; Korte & Lin, 2013)

11

Literature ReviewAcademics, industry, and government agencies own this shared problem, and it requires a systemic solution. Unfortunately, we know little about ¨ Engineering practice for freshout engineers.

(Kowtha, 2008; Trevelyan, 2007, 2008; Brunhaver et al., 2010, in press; Winters et al., 2013)

¨ What freshout engineers do successfully and unsuccessfully in the workplace. (Trevelyan & Tilli, 2008; Trevelyan ,2008, 2009)

¨ Barriers to desired workplace performance.(Korte et al., 2008; Atman et al., 2010; Brunhaver et al., 2010, in press; Anderson et al., 2010)

E2R2P is an opportunity to collaborate systemically to decrease ramp-up time to competent performance.

12

Thanks to Our Professional and Industry Sponsors!¨ Practicing

engineers at ISPE¨ BSU COEN

Advisory Council

¨ Focus Groups at local engineering firms

13

MethodPracticing Engineer Survey

Short survey measuring:¨ Types of work assigned to

freshouts.¨ Typical time to competence¨ Costs and risks that

organizations incur when freshouts can’t perform to standards

¨ Typical project organization for groups of engineers

¨ Organizational support for freshouts ISPE (2012), n = 23

14

Time to Competency for "Fresh-Outs" (in months)

Six to Nine Ten to Twelve Thirteen to Sixteen

Seventeen to Twenty-Four

More than Twenty-Four

2 3 4 4

10

15

Typical Assignment Size

Desig

n

Analys

is

Drawin

g Dra

fts

Proje

ct M

anag

emen

t

2 2

6

2

Large/Complex Projects

Desig

n

Analys

is

Drawin

g Dra

fts

Proje

ct M

anag

emen

t

1914

19

3

Small/Simple Projects

16

Organizational Support

Formal

Tra

inin

g

Formal

Men

torin

g

Info

rmal

Men

torin

g

Orienta

tion

Perfo

rman

ce F

eedbac

k

SOPs

Other

9 8

1511

20

10

3

17

MethodFocus Groups

Collect Incidents

Generate Categories

Negotiate Categories

Group Incidents under Categories

Select Incidents of Non-performance

Group Incidents Under a Root CauseRank Categories

Critical Incident Technique(Flanagan, 1954)

Nom

inal Group Technique

(Delp et al., 1975)

Cause Analysis

18

Collect IncidentsCritical Incident Technique (Flanagan, 1954)

19

Focus Group ResultsPerformance Categories

Category Unsuccessful SuccessfulCommunication 9 8Technical Fundamentals 3 11

Business Systems

5 7

Design 5 6Motivation 6 5Problem Solving 6 5Initiative 2 8Positive Attitude 3 4Work Ethic 2 5Circuit Debug 3 3

Freshout-Defined Categoriesn = 10

Competency Unsuccessful SuccessfulDesign 14 18Communication 18 10Analysis 10 14Motivation 8 10Technical Fundamentals 3 12

Problem Solving 7 6Software 3 10Business Systems 5 7

Initiative 2 8Leadership 4 5Process Knowledge 4 3

Positive Attitude 3 4

Manager-Defined Categoriesn = 20

20

Focus Group ResultsConsequences

Successful Performance Consequences

# of Events

Met Schedule 19Freed up Senior Staff 18Saved Resources 18Improved Processes 15Developed New Tool 14Developed Skill / Knowledge 12Saved Time 12Gained Client Confidence 10Gained Employer Confidence

10

Stayed Within Budget 7Increased Productivity 6

Unsuccessful Performance Consequences

# of Events

Lost Time 34Rework 22Additional Staff Support 17Missed Deadline 12Increased Costs 10Wasted Budget 8Lost Employer Confidence 5Stressed Staff 5Job Unfinished 4Lost Client Confidence 4Exceeded Budget 3

Aggregated Freshout and Management Events

21

Categories, Activities and Consequences

Top-Ranked Categories Source Activity Consequence

1. Communication / Teamwork

Freshout “The first time I had to write up anengineering report--I struggled doing it. Inever had to submit something to IDQbefore, and I wasn’t confident”

“It just took me a lot of my personal timeand a senior had to review it and it wasn’tgood.”

2. Skill Develop-ment / Learning

Freshout “I observed a lot of nuclear test proceduresand started evolutions on board and navalvessels so after you learned we went out tothe vessels and basically were like buddybuddy with a qualified test engineer to learnthe procedures and observe what was goingon…”

“I think I was pretty successful. You had toregurgitate what you learned. You had aqualification check off sheet and so you’d haveto go back with that test engineer andregurgitate what you saw and how you thoughtthe procedure went then he would sign off your sheet and you’d go on to the next.”

3. Work Ethic Manager “We had a junior engineer. I think he hadonly been here a couple of weeks. Got himonto a team for ongoing projects and gavehim minimal amount of direction…”

“…He immediately started contributing morethan I think any of us envisioned…within afew days he was coordinating with severalother staff inside and outside our company.”

4. Business Systems / Processes

Freshout “I didn’t consider that activity [resolving drawing issues] successful because I was not aware or did not have enough knowledge about manufacturing processes in general to really be effective and resolve those issues in a short time span. It took me a lot longer than it could have...”

“As a result it there were a lot of late deliveries to those revisions and caused us to do multiple rework and multiple parts…”

22

Categories, Activities and Consequences (con’t)

Top-Ranked Categories Source Activity Consequence5. Problem Solving Manager “There was an issue that was found on the

floor and it was in the CAB. The CAB group lead came to this person [the new engineer] and asked him to go figure out what the problem was.”

“That person went out there, they figured it out, they investigated it…found out that it was in fact a design error, and they went and took care of it…ended up fixing the design error correctly…It allowed production to keep going and we had minimal down time”

6. Analysis Freshout “The engineer that was doing it [analysis] had a lot going on. Field work and stuff so [he]… passed it off to me. “

The biggest impact was probably time and cost for doing it slower than he was or would’ve and … a couple hours of coaching…So there was a learning curve to it..”

7. Technical Fundamentals

Manager “…basically he [the new engineer] didn’t know how to use the software and didn’t have the general, multiple CAD system-type training.

“He was focused all on one CAD software whenhe was in school and so although he couldmodel in 3-D, he couldn't psych out how thissoftware thought and how it behaved and thatit was different than what he had donebefore…”

8. Design Manager “I had a young engineer who was tasked with developing a draw bar for semi trailers.”

“He [the new engineer] was able to, in about afour-month period, generate a single CADmodel that could then be driven through afamily table to automatically generate newdesigns and drawings. He took a typically 8hour to 12-hour project down to 30 minutes.The impact was huge.”

23

Categories, Activities and Consequences (con’t)

Top-Ranked Categories Source Activity Consequence9. Software Freshout “We had switched over to this other software

and for 9 months I don’t think we produced any usable products. “

“.Literally everything we did in the software for six months was never used. It was never useable. Had to be trashed. It was an extremely frustrating part of my career. “

11. Leadership [When I joined the design team] “I wasimmediately put in charge of an entire CABdesign for one of our contracts. Essentiallywhat that entails is facilitating the whole

designprocess.”

“I don’t feel like I was ill prepared to do that.”

24

Root Cause AnalysisInstrumentation

• Data• Expectations• Feedback• Standard Operating

Procedures

• Resources• Software• Tools• Support

• Incentives• Rewards• Consequences

• Knowledge• Skills

• Physical Capacity• Mental Capacity• Flexibility• Resilience

• Motives• Affect• Work Habits• Drive

EN

VIR

ON

ME

NT

PE

RS

ON

INFORMATION TOOLS MOTIVATION

25

Root Cause AnalysisResults

10

2

24

8

11

Managers

InformationToolsIncentivesKnolwedgeCapacityMotivation

6

7

22

1 2

Freshouts

26

Root Cause AnalysisResults

16

9

46

8

13

Combined

InformationToolsIncentivesKnolwedgeCapacityMotivation

35%

29%

11%

11%

8%6%

Dean (1997)

27

Limitations

¨ Validity and reliability of the Practicing Engineer Survey is unknown.

¨ Small exploratory study using a convenience sample of local engineering firms.

¨ No post-focus group data checking with participants and their company sponsors (managers).

28

Conclusions

¨ Decreasing time to engineering workplace competency is a shared problem.

¨ Freshout engineers are variably prepared to enter the workplace.

¨ The engineering workplace supports freshout performance variably well.

¨ Socialization and onboarding involves a lot of self-reported learning.

¨ We don’t know about the extent to which fixing the workplace environment and introducing it to students sooner would decrease ramp-up time.

29

Next Steps¨ Seek funding to expand research.

– Include other engineering populations.– Regional, national, or international sample.– Scale up and automate processes.

¨ Investigate research questions about:– Blurring traditional academic and industry

boundaries.– The extent to which a smarter workplace

environment introduced in academics could decrease ramp-up time.

¨ Create a collaborative venue to decrease ramp-up time.

30

Summary

¨ Research questions and importance¨ Literature review¨ Method¨ Results¨ Limitations¨ Conclusions

What are your lessons learned?

How might you apply them back on the job?

31

ReferencesAnderson, K., Courter, S., McGlamery, T., Nathans-Kelly, T., & Nicometo, C. (2009, June). Understanding the

current work and values of professional engineers: Implications for engineering education. Paper presented at the American Society for Engineering Education, Austin, TX.

Anderson, K.J.B., Courter, S.S., McGlamery, T., Nathans-Kelly, T.M., & Nicometo, C.G. (2010). Understanding engineering work and identity: A cross-case analysis of engineers within six firms. Engineering Studies, 2(3), 153-174. doi: 10.1080/19378629.2010.519772

ASEE & NSF. (2013). Transforming undergraduate education in engineering: Phase I--Synthesizing and integrating industry perspectives. Arlington, VA: American Society for Engineering Education. Retrieved from http://www.asee.org/TUEE_PhaseI_WorkshopReport.pdf

Atman, C.J., Sheppard, S.D., Turns, J., Adams, R.S., Fleming, L.N., Stevens, R., . . . Lund, D. (2010). Enabling engineering student success: The final report for the center for the advancement of engineering education. San Rafael, CA: Center for the Advancement of Engineering Education. Retrieved from http://www.engr.washington.edu/caee/CAEE%20final%20report%2020101102.pdf

Blom, A., & Saeki, H. (2011). Employability and skill set of newly graduated engineers in India. The World Bank South Asia Region Education Team. Retrieved from http://elibrary.worldbank.org/docserver/download/5640.pdf?expires=1337278148&id=id&accname=guest&checksum=E557A65C13E9CA650433F9E5798C4242

Borrego, M., & Bernhard, J. (2011). The emergence of engineering education research as an internationally connected field of inquiry. Journal of Engineering Education, 100(1), 14-47.

Bradt, G.B., & Vonnegut, M. (2009). Onboarding: How to get your new employees up to speed in half the time. Hoboken, NJ: John Wiley & Sons.

Brown, T. (2005). Strategy by design. Fast Company, (June 1). Retrieved from http://www.fastcompany.com/52795/strategy-design

Brunhaver, S., Korte, R., Lande, M., & Sheppard, S. (2010, June). Supports and barriers that recent engineering graduates experience in the workplace. Paper presented at the American Society for Engineering Education, Vancouver, British Columbia.

Brunhaver, S.R., Korte, R., Barley, S.R., & Sheppard, S.D. (in press). Bridging the gaps between engineering education and practice.

Bucciarelli, L.L., & Kuhn, S. (1997). Engineering education and engineering practice: Improving the fit. In S. R. Barley & J. E. Orr (Eds.), Between craft and science: Technical work in the United States (pp. 210-229). Ithica, NY: Cornell University Press.

Dai, G., & De Meuse, K.P. (2007). A review of onboarding literature. Minneapolis, MN: Lominger Limited. Retrieved from http://boardoptions.com/onboardingevidence.pdf

32

ReferencesDean, P.J. (1997). Thomas f. Gilbert, ph. D: Engineering performance improvement with or without training. In P. J.

Dean & D. E. Ripley (Eds.), Performance improvement series: Vol. 1. Performance improvement pathfinders: Models for organizational learning systems (pp. 45-64). Silver Spring, MD: International Society for Performance Improvement.

Delp, P., Thesen, A., Motiwalla, J., & Seshardi, N. (1977). Nominal group technique. In P. Delp (Ed.), Systems tools for project planning (pp. 14-18). Bloomington, IN: International Development Institute.

Duderstadt, J.J. (2010). Engineering for a changing world. In D. Grasso & M. Burkins (Eds.), Holistic engineering education: Beyond technology (pp. 17-35). New York, NY: Springer. Retrieved from http://milproj.dc.umich.edu/pdfs/2009/Engineering%20for%20a%20Changing%20World.pdf

Flanagan, J.C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-358. doi: 10.1037/h0061470

Grant, C.D., & Dickson, B.R. (2006). Personal skills in chemical engineering graduates: The development of skills within degree programmes to meet the needs of employers. Education for Chemical Engineers, 1(1), 23-29. doi: http://dx.doi.org/10.1205/ece.05004

Hoey, J.J., & Gardner, D.C. (1999). Using surveys of alumni and their employers to improve an institution. New Directions for Institutional Research, 1999(101), 43-59. doi: 10.1002/ir.10103

Jonassen, D., Strobel, J., & Chwee Beng, L. (2006). Everyday problem solving in engineering: Lessons for engineering educators. Journal of Engineering Education, 95(2), 139-151. doi: 10.1002/j.2168-9830.2006.tb00885.x

Jones, K. (2013). Virtual onboarding for today’s global workforce. Oakland, CA: Bersin by Deloitte. Retrieved from http://communication.on24.com/virtual-onboarding-for-todays-global-workforce

Korte, R., & Lin, S. (2013). Getting on board: Organizational socialization and the contribution of social capital. Human Relations, 66(3), 407-428. doi: 10.1177/0018726712461927

Korte, R., Sheppard, S., & Jordan, W. (2008). A qualitative study of the early work experiences of recent graduates in engineering. Paper presented at the American Society for Engineering Education, Pittsburgh, PA.

Kowtha, N.R. (2008). Engineering the engineers: Socialization tactics and new engineer adjustment in organizations. IEEE Transactions on Engineering Management, 55(1), 67-81. doi: 10.1109/TEM.2007.912809

33

ReferencesLindo, D.K. (2010). New employee orientation is your job! SuperVision, 71(9), 11-15. McCrohon, M., & Gibson, P. (2009, December). Student experiences in the direct applicability of their engineering

education to professional practice. Paper presented at the 20th Annual Conference for the Australasian Association for Engineering Education, Adelaide, Australia.

Montesano, A. (2007). Orienting new employees for career success: Effective onboarding ensures smoother integration and faster productivity. Canadian HR Reporter, 20(18), 23-24.

Morgan, M., & O’Gorman, P. (2010, July). Developing industry-ready engineers: A regional university perspective. Paper presented at the International Conference on Engineering Education ICEE-2010, Gliwice, Poland.

National Academy of Engineering. (2005). Educating the engineer of 2020: Adapting engineering education to the new century. Washington, DC: National Academies. Retrieved from http://www.nap.edu/catalog.php?record_id=11338#toc

Passow, H.J. (2012). Which ABET competencies do engineering graduates find most important in their work? Journal of Engineering Education, 101(1), 95-118. doi: 10.1002/j.2168-9830.2012.tb00043.x

Ramlall, S. (2004). A review of employee motivation theories and their implications for employee retention within organizations. The Journal of American Academy of Business, Cambridge 5(1/2), 52-63.

Roethle, J. (2012). Developing a new employee onboarding program in a small engineering department. Milwaukee School of Engineering, Milwaukee, WI.

Rollag, K., Parise, S., & Cross, R. (2005). Getting new hires up to speed quickly. MIT Sloan Management Review, 46(2), 35-41.

Sheppard, S., Macatangay, K., Colby, A., & Sullivan, W.M. (2009). Educating engineers: Designing for the future of the field (Vol. 9): Jossey-Bass San Francisco, CA.

Snell, A. (2006). Researching onboarding best practice: Using research to connect onboarding processes with employee satisfaction. Strategic HR Review, 5(6), 32-35.

Spinks, N., Silburn, N., & Birchall, D. (2006). Educating engineers for the 21st century: The industry view. London: The Royal Academy of Engineering.

Stump, G.S., Hilpert, J.C., Husman, J., Chung, W.-t., & Kim, W. (2011). Collaborative learning in engineering students: Gender and achievement. Journal of Engineering Education, 100(3), 475-497.

34

References

Trevelyan, J. (2007). Technical coordination in engineering practice. Journal of Engineering Education, 96(3), 191-204.

Trevelyan, J. (2008). A framework for understanding engineering practice. Paper presented at the American Society for Engineering Education, Pittsburgh, PA.

Trevelyan, J., & Tilli, S. (2008). Longitudinal study of Australian engineering graduates: Perceptions of working time. Paper presented at the American Society for Engineering Education, Pittsburgh, PA.

Trevelyan, J.P. (2009). Engineering education requires a better model of engineering practice. Paper presented at the Proceedings of the Research in Engineering Education Symposium, Palm Cove, QLD, Australia.

Trevelyan, J.P. (2010). Mind the gaps: Engineering education and practice. Paper presented at the 21st Annual Conference for the Australasian Association for Engineering Education, Sydney, Australia.

Winters, K.E., Matusovich, H.M., Brunhaver, S., Chen, H.L., Yasuhara, K., & Sheppard, S. (2013, June). From freshman engineering students to practicing professionals: Changes in beliefs about important skills over time. Paper presented at the American Society for Engineering Education, Atlanta, GA.