Student Outcomes Manual - Bryan College · Student Outcomes Manual 2018-2019 ... ETS MAJOR FIELD TEST BUSINESS 2013 - 2019 TOTAL STUDENTS: 136 seniors Mean scores are shown in parentheses;

1

Student Outcomes Manual

2018-2019

This document provides up-to-date assessment information to the stakeholders of Bryan College regarding

the extent to which the college’s educational goals and student learning outcomes have been met.

2

Office of Institutional Effectiveness

TABLE OF CONTENTS

Bryan College Educational Goals ................................................................................................... 3

Bryan College General Education ................................................................................................... 5

Bryan Freshman Mean ACT Scores............................................................................................... 7

ETS Proficiency Profile: General Education Learning Outcomes, National Percentiles (New

Students and Seniors) ....................................................................................................................... 8

ETS Proficiency Profile: Proficiency Classification (Seniors)....................................................... 9

ETS Major Field Test: Biology ..................................................................................................... 10

ETS Major Field Test: Business ................................................................................................... 11

Discipline Content ACAT Exam: History .................................................................................... 12

ETS Major Field Test: Literature in English ............................................................................... 13

ETS Major Field Test: Mathematics ............................................................................................. 14

ETS Major Field Test: Music ......................................................................................................... 15

ETS Major Field Test: Psychology ................................................................................................ 16

PRAXIS II Scores: Education Licensure Majors ......................................................................... 17

SSI: How Satisfied Are Bryan Students Compared to Students in Private Colleges? ............. 18

SSI: Student Evaluations of Majors and Advisors ...................................................................... 19

Student Evaluations of Majors and Advisors (from in-house surveys) ...................................... 20

Institutional Mission Expressed as Educational Goals: Class of 2019 as New Students and

Seniors .............................................................................................................................................. 21

General Education Goals: New Students (2015) & Seniors (2019) ............................................. 22

Alumni Survey: 2014-2018 ............................................................................................................. 23

Alumni Employment: 2014-2018 ................................................................................................... 24

3

Academic Year: 2018-2019

College Mission:

The mission of Bryan College is to educate students to become servants of Christ to make a difference in today's

world. The College seeks to assist in the personal growth and development of qualified students by providing an

education based upon an integrated understanding of the Bible and the liberal arts.

Educational Goals Assessment Procedures & Instruments

1. Students will develop a knowledge of the Bible

and the liberal arts and an ability to harmonize

that knowledge through an understanding of their

relationships.

Student Achievement Criteria

Student Satisfaction Inventory

ETS Proficiency Profile

Senior Oral Examinations, Presentations, and/or Recitals

Alumni Surveys

Other Assessments

Student/Peer Course Evaluations

General Education Learning Outcomes Survey

Student Satisfaction Survey

Major Surveys

2. Students will develop competency in one or more

subjects as a foundation for graduate study or

vocations related to the disciplines of Christian

Studies, Humanities, Natural Science, and Social

Science.


Alumni Surveys


Praxis II, ETS Major Field Tests, ACAT History Exam

Senior Oral Examinations & Performance Platforms

Field Experience & Internship Evaluations

Other Assessments


Educational Goals Survey



3. Students will demonstrate academic excellence by

thinking critically, working independently and

cooperatively, communicating clearly, and

expressing themselves creatively.


Alumni Surveys


Praxis II, ETS Major Field Tests, ACAT History Exam

Senior Oral Examinations & Performance Platforms


Other Assessments


Student Surveys


Educational Goals

Student Learning Outcomes

4

Educational Goals Assessment Procedures & Instruments



4. Students will develop wholesome attitudes,

healthful habits, responsible citizenship,

constructive interests and skills, and the

recognition that education is a continuing process.

Student Assessment Criteria

Alumni Surveys



Other Assessments

Spiritual Formation Survey



Evaluations of Chapels

Conference Evaluations

Leadership Training Evaluations

5. Students, faculty, and staff will serve the local

community and the Bryan constituency with

academic and creative experiences and

consultative services in ways consistent with the

educational philosophy, purpose, and resources of

the College.


Alumni Surveys


Other Assessments




PCI Evaluations


6. Students will mature spiritually by engaging in

opportunities for Christian leadership, ministry,

service, worship, and discipleship.


Alumni Surveys


Field Experience and Internship Evaluations

Other Assessments

MLK Service Evaluations

Student Surveys



Evaluations of Chapels

PCI Evaluations

CLF Evaluations

Spiritual Life Conference Evaluations

7. Students will enhance their undergraduate and

graduate education by participation in research

projects and colloquia conducted by Bryan’s

faculty, academic departments, and affiliate and

ancillary organizations.


Research Projects, Written/Oral Reports, Presentations,

Recitals

Other Assessments




5

Academic Year: 2018-2019

College Mission:

The mission of Bryan College is to educate students to become servants of Christ to make a difference in today's

world. The College seeks to assist in the personal growth and development of qualified students by providing an

education based upon an integrated understanding of the Bible and the liberal arts.

Learning Outcomes Assessment Procedures & Instruments

1. Students will make a difference in today’s

world by strengthening their educational

foundation.


ETS Major Field Tests, ACAT History Exam

Alumni Surveys


Other Assessments

Developmental Coursework Evaluation

Freshman English Course Evaluation

General Education Math Course Evaluation





world by developing a Christian Worldview.


Alumni Surveys


PEERS Test

Other Assessments

Christian Worldview Course Evaluation

Biblical Foundations Course Evaluation

Pentateuch Course Evaluation

New Testament Literature Course Evaluation

Christian Life Formation Course Evaluation




Curriculum Mapping

Practical Christian Involvement (PCI) Survey

PCI Evaluation

GENERAL EDUCATION

LEARNING OUTCOMES

6

Learning Outcomes Assessment Procedures & Instruments


world by engaging the culture.


Alumni Surveys


Other Assessments

General Education Psychology Course Evaluation

Introduction to Communications Course Evaluation

Foreign Language Course Evaluation





world by exploring the arts.


Alumni Surveys


Other Assessments

Introduction to Literature Course Evaluation

Introduction to Fine Arts Course Evaluation





world by understanding the natural world.


Alumni Surveys

ETS Proficiency Profile Lab Sciences Course Evaluation

Other Assessments

Non-lab Sciences Course Evaluation

Concepts of Physical Fitness Course Evaluation

Exercise and Health Activities Course Evaluation




7

2018 National Average: 20.8 (58th percentile)

2018 Tennessee Average: 20.2 (51st percentile)

Four-Year National Average: 20.9

Four-Year Bryan Average: 22.7

Six-Year Bryan Average: 22.9

6871

6469

6569

0

10

20

30

40

50

60

70

80

90

100

2013 (23.0) 2014 (22.9) 2015 (22.5) 2016 (22.9) 2017 (22.6) 2018 (23.3)

BRYAN FRESHMAN

MEAN ACT SCORES

2005 - 2010

Nat

ion

al P

erce

nti

le

8


The charts below indicate how Bryan College students performed nationally on the Proficiency Profile.

These percentiles represent the percentage of students in liberal arts colleges that scored at or below

the raw scores received by students at Bryan College.

GENERAL EDUCATION LEARNING OUTCOMES

NATIONAL PERCENTILES (New Students & Seniors)

54

71

56

40

71

40

7469

0

10

20

30

40

50

60

70

80

90

100

Na

tio

na

l P

erce

nti

le

Senior (SP-19) Percentiles

79 79 77

34

85

59

87

75

0

10

20

30

40

50

60

70

80

90

100

Na

tio

na

l P

erce

nti

le

Freshman (FA-18) Percentiles

9


The charts below indicate how Bryan College students performed nationally on the Proficiency Profile.

These scores represent the performance of Bryan freshmen and senior students in various liberal arts

categories (possible range from 100-130).

GENERAL EDUCATION LEARNING OUTCOMES

NATIONAL PERCENTILES (New Students & Seniors)

Mean score for New Students = 439.16 (68%ile nationally)

Mean score for Seniors = 449.38 (66%ile nationally)

New Students Seniors

Value Added Value Added

Critical

Thinking

2.1 2.9

Reading 2.7 3.1

Writing 2.1 2.5

Mathematics 1.9 3.0

Humanities 1.9 2.8

Social

Sciences

2.1 2.7

Natural

Sciences

2.2 2.6

111.08

117.74

114.02

111.32

114.70

111.86

115.30

112.72

120.11

115.93

113.70

116.71

113.59

117.46

100 105 110 115 120 125 130

Critical Thinking

Reading

Writing

Mathematics

Humanities

Social Sciences

Natural Sciences

Seniors New Students

10

ETS MAJOR FIELD TEST

BIOLOGY

2013 - 2019

TOTAL STUDENTS, 42 seniors

*2015 is combined with 2016: 3 students took the exam in 2015

**2017 is combined with 2016: 4 students took the exam in 2017

***2018 is combined with 2017: 4 students took the exam in 2018

Mean scores are shown in parentheses; the range of scores is 120-200.

Seven-Year Bryan Average: 160.4

94 95

84

72 72

88

31

0

10

20

30

40

50

60

70

80

90

100

2013 (165)

6 students

2014 (165)

8 students

2015-2016

(160)

8 students*

2016 (158)

5 students

2016-2017

(165)

9 students*

2017-2018

(161)

8 students**

2019 (149)

12 students

Na

tio

na

l P

erce

nti

le

11


BUSINESS

2013 - 2019

TOTAL STUDENTS: 136 seniors

Mean scores are shown in parentheses; the possible range of scores is 120-200.


8589

97

76

94

70

93

0

20

40

60

80

100

120

2013 (158)

19 students

2014 (160)

15 students

2015 (163)

16 students

2016 (156)

14 students

2017 (160)

22 students

2018 (154)

24 students

2019 (160)

26 students

Na

tio

na

l P

erce

nti

le

12

DISCIPLINE CONTENT ACAT EXAM

HISTORY

2013 - 2019


Year Individual

Test Scores

Average

Scores

National

Percentile

2013 *not available* 509 54

2014 *not available* 568 75

2015 711 600 84

664

574

548

504

2016 654 572 76

562

499

2017 571 518 57 536 497 469

2018 502 449 25*

394

2019 604 547 65*

490

*ACAT does not provide group scores for samples of fewer than 3 students.

Mean scores are shown in parentheses; the range of scores is

200-800 for ACAT exams.

13


LITERATURE IN ENGLISH

2013- 2019


*2018 (3 students) is combined with 2017 (1 student) and 2016 (5 students)

**2017 (1 student) is combined with 2016 (5 students)

Mean scores are shown in parentheses; the range of scores is 120-200.


98 99

87

60

69

84

99

0

10

20

30

40

50

60

70

80

90

100

2013 (178)

6 students

2014 (181)

8 students

2015 (165)

16 students

2016 (156)

5 students

2016-2017

(158)

6 students

2016- 2018

(163) 9

students **

2019 (181)

2 students

Nat

ion

al P

erc

en

tile

14

ETS MAJOR FIELD EXAM

MATHEMATICS

2013 - 2019

TOTAL STUDENTS: 11

Year Individual Test

Scores

Average

Scores

National

Percentile

2013 142 165.5 90th

189

2014 154 159 66th

164

2015 151 152.5 38th

154

2016 *No students were tested. ********* ****

2017 142 142 6th

2018 *No students were tested. ********* ****

2019 132 145

7th

138

138

171

National percentiles calculated for years 2013-2018 based on

Comparative Data provided by ETS.

15


MUSIC

2013 - 2019

TOTAL STUDENTS: 37 students

**2018 (2 students) combined with 2017 (6 students)

Mean scores are shown in parentheses; the range of possible scores is 120-200.

Seven-Year Bryan Average: 155

*Note: Scores included in this data represent students participating in the following music major

options: Music Performance, Piano Pedagogy, Music Education, and Church Music/Music

Ministry.

90

7874

48

78

48

38

0

10

20

30

40

50

60

70

80

90

100

2013

(158.75)

4 students

2014 (159)

3 students

2015 (155)

8 students

2016 (151)

6 students

2017 (157)

6 students

2018 (149)

8 students**

2019 (147)

8 students

Nat

ion

al P

erc

en

tile

16


PSYCHOLOGY

2013 – 2019*


Mean scores are shown in parentheses; the range of possible scores is 120-200.

*Note: In the 2014 version, dates were updated to reflect the academic year during which

the test was administered.


9591

94

76

91

83

92

0

10

20

30

40

50

60

70

80

90

100

2013 (174)

14 students

2014 (167)

19 Students

2015 (172)

15 students

2016 (168)

8 students

2017 (168)

7 students

2018 (164)

15 students

2019 (168)

10 students

Nat

ion

al P

erc

en

tile

17

PRAXIS II SCORES

EDUCATION LICENSURE MAJORS

100 100 100 100 100100 100 100 100 100

0

20

40

60

80

100

120

K-6 Principles ofLearning and

Teaching

7-12 Principles ofLearning and

Teaching

Curriculum,Instruction, and

Assessment

Elementary ContentKnowledge

Teaching Reading

Perc

ent

2016-2017 and 2017-2018

Pass Rate 2016-2017 Pass Rate 2017-2018

175

171 171

165

177176

179

175 174 174

160

157

153

163 162

140

145

150

155

160

165

170

175

180

185

K-6 Principles ofLearning and

Teaching

7-12 Principles ofLearning and

Teaching

Curriculum,Instruction, and

Assessment

Elementary ContentKnowledge

Teaching Reading

Pra

xis

Test

Sco

re R

ange

2016-2017 and 2017-2018

BC Mean 2016-2017 (Raw Score) BC Mean 2017-2018 (Raw Score)2 TN Passing Score

18

STUDENT SATISFACTION SURVEY (SSI)

How Satisfied Are Bryan Students Compared to

Students in Private Colleges

November 2018

(77 of 312 Students Surveyed – 25% of Sample Group)

5.62

5.44

5.77

5.58

6.01

5.99

5.66

5.15

5.94

5.62

5.28

5.62

5.48

5.11

5.59

5.25

5.61

5.64

5.27

5.06

5.44

5.35

5.36

5.42

1.00 2.00 3.00 4.00 5.00 6.00 7.00

Student Centeredness

Campus Life

Instructional Effectiveness

Recruitment and Financial Aid

Campus Support Services

Academic Advising

Registration Effectiveness

Safety and Security

Concern for the Individual

Service Excellence

Responsiveness to Diverse Populations

Campus Climate

Other Private College Students Bryan Students

Not satisfied at all Very satisfied

19

STUDENT EVALUATIONS OF MAJORS

AND ADVISORS, 2012-2018

(from Student Satisfaction Inventory)

685 Students Responding Over 7 Years

CONTENT OF COURSES WITHIN THE MAJORS

ACADEMIC ADVISING

5.84 5.97 5.90 5.935.66 5.64

5.87

1

2

3

4

5

6

7

2012

(127 students)

2013

(105 students)

2014

(162 students)

2015

(80 students)

2016

(77 students)

2017

(65 students)

2018

(77 students)

6.12 6.19 6.12 6.12 6.105.80

5.99

1

2

3

4

5

6

7

2012

(127 students)

2013

(127 students)

2014

(62 students)

2015

(80 students)

2016

(77 students)

2017

(65 students)

2018

(77 students)

Very Satisfied

Somewhat Satisfied

Somewhat Dissatisfied

Not Satisfied at All

Very Satisfied

Somewhat Satisfied

Somewhat Dissatisfied

Not Satisfied at All

20

STUDENT EVALUATIONS OF MAJORS

AND ADVISORS, 2010-2014 & 2018 (from in-house

surveys)

QUALITY OF COURSEWORK IN THE MAJORS

QUALITY OF ACADEMIC ADVISING

*The gap in data reflects a period in which the SSI Item #8 was considered adequate for an

assessment of Academic Advising. More recent institutional reflection has recognized the

usefulness of an additional internal measure on this item, starting in 2018.

3.44 3.45 3.523.41

3.57

3.10 3.05

1

2

3

4

2010

(298 students)

2011

(222 students)

2012

(168 students)

2013

(213 students)

2014

(180 students)

2018

(309 students)

2019

(328 students)

3.36

3.80

3.54 3.50 3.56

3.08 3.09

1

2

3

4

2010

(298 students)

2011

(222 students)

2012

(168 students)

2013

(213 students)

2014

(180 students)

2018

(309 students)

2019

(328 students)

(4) Excellent

(3) Good

(2) Fair

(1) Poor

(4) Excellent

(3) Good

(2) Fair

(1) Poor

21

INSTITUTIONAL MISSION EXPRESSED AS

EDUCATIONAL GOALS:

CLASS OF 2018 AS NEW STUDENTS AND SENIORS

New Students, August, 2015 (39 students out of 234 possible –17%)

Seniors, April, 2019 (74 seniors out of 192 possible – 39%)

Degree to Which the Class of 2019 Stated They Achieved Bryan’s Mission and

Educational Goals, as Both New Students and Seniors

Educational Goal #1 (Value Added: +0.09)

“Students will develop a knowledge of the Bible and the liberal arts and an ability to harmonize that knowledge

through an understanding of their relationships”


“Students will develop competency in one or more subjects as a foundation for graduate study or vocations

related to the disciplines of Christian Studies, Humanities, Natural Science, and Social Science”

Educational Goal #3 (Value Added:+0.03)

“Students will demonstrate academic excellence by thinking critically, working independently and

cooperatively, communicating clearly, and expressing themselves creatively”

Educational Goal #4 (Value Added: 0)

“Students will develop wholesome attitudes, healthful habits, responsible citizenship, constructive interests

and skills, and the recognition that education is a continuing process”


“Students, faculty, and staff will serve the local community and the Bryan constituency with academic and

creative experiences and consultative services in ways consistent with the educational philosophy, purposes,

and resources of the College”


“Students will mature spiritually by engaging in opportunities for Christian leadership, ministry, service,

worship, and discipleship”


“Students will enhance their undergraduate and graduate education by participation in research projects and

colloquia conducted by Bryan’s faculty, academic departments, and affiliate and ancillary organizations”

Mission Statement (Value Added: +0.03)

“Educating students to become servants of Christ to make a difference in today’s world”

2.03 1.94

2.25 2.27

1.971.8

1.92

2.38

2.12 2.112.28 2.27

2.4 2.4

2.14

2.41

0

0.5

1

1.5

2

2.5

3

Goal #1 Goal #2 Goal #3 Goal #4 Goal #5 Goal #6 Goal #7 Misson

New Students Seniors

Extensively

Moderately

Minimally

Not at all

22

GENERAL EDUCATION GOALS:

NEW STUDENTS (2015) AND SENIORS (2019)

New Students, August, 2014 (123 students out of 193 possible – 64%)

Seniors, April, 2018 (39 seniors out of 100 possible – 39%)

Degree to Which Class of 2018 Stated They Achieved

Bryan’s General Education Learning Outcomes

“Students will make a difference in today’s world by…”

Learning Outcome #1: “…strengthening their foundation” (Value Added: +0.03)

(reading, writing, mathematics)

Learning Outcome #2: “…developing a Christian worldview” (Value Added: +1.12)

(biblical studies)

Learning Outcome #3: “…engaging the culture” (Value Added: +0.26)

(social sciences, communications, languages)

Learning Outcome #4: “…exploring the arts” (Value Added: +0.35)

(literary, visual, performing arts)

Learning Outcome #5: “…understanding the natural world” (Value Added: +0.16)

(natural sciences)

(3) Achieved

Extensively

(2) Achieved

Moderately

(1) Achieved

Only

Minimally

(0) Did Not

Achieve

At All

2.33

1.47

1.771.68

1.89

2.36

2.59

2.03 2.03 2.05

0

0.5

1

1.5

2

2.5

3

As New Students

As Seniors

23

ALUMNI SURVEY

2014-2018

Number of Alumni Employed, Attending Graduate School, or

both Employed and in Graduate School

45 46

28 27 30

8 8

139

11

4 5

1

1

2

0

10

20

30

40

50

60

70

2014 (61) 2015 (68) 2016 (48) 2017 (38) 2018 (44)

Per

cen

tage

of

Stu

den

ts

Year of Graduation

Employed Graduate School and Employed Graduate Schoool

24

ALUMNI STATISTICS

Percentage of Bryan Alumni employed or pursuing additional

education within six months of graduation compared to National

Association of Colleges and Employers

2014-2018

*NACE 2018 data will be available Fall 2019.

93.5

86.8 87.5

97.4 97.7

80.3 78.9 81 81.2

0

20

40

60

80

100

120

2014 2015 2016 2017 2018

Per

cen

tage

s

Graduation Year

Bryan Alumni NACE

Documents

Student Outcomes Manual - Bryan College · Student Outcomes Manual 2018-2019 ... ETS MAJOR FIELD TEST BUSINESS 2013 - 2019 TOTAL STUDENTS: 136 seniors Mean scores are shown in parentheses;