Phil Winne Simon Fraser University. 1.what is robust learning? 2.what does learning science offer? 3.self-regulated learning (SRL) 4.tools for researching

Phil Winne Simon Fraser University

1.what is robust learning? 2.what does learning science offer? 3.self-regulated learning (SRL) 4.tools for researching learning & SRL 5.traces keys to modeling SRL 6.learning analytics 7.nurturing a community of practice, an ecology of experimentation 8.discussion does this resonate with you? Colorado State University 2

3 LASI 2014 Colorado State University

learning is robust if acquired knowledge or skill exhibits long-term retentionexhibits long-term retention at least days, perhaps years transferstransfers used in situations differing from the situation when instructed accelerates future learningaccelerates future learning adapted from: http://learnlab.org/research/wiki/index.php/robust_learning Colorado State University 4

a process by which an individual productively adapts to stress & adversity by developing realistic plansdeveloping realistic plans realistically estimating self efficacyrealistically estimating self efficacy taking steps / following through with the plantaking steps / following through with the plan communicating & problem-solving as neededcommunicating & problem-solving as needed exercising self-control managing impulses & feelingsexercising self-control managing impulses & feelings nurturing positive self-conceptnurturing positive self-concept Colorado State University 5

Facet of MotivationWhat a learner wants to know (or estimate) outcome expectationwhat result does a particular operation produce? efficacy expectationcan I carry out the operation(s) to achieve that result? incentivewhats the value of achieving that result? attributionwhat explains my successes & failures? utilityis the overall benefit worth the risk/cost? Colorado State University 6 rearrange the letters: AEIOU

temperament + skill to inhibit impulses & avert disruptive behaviorinhibit impulses & avert disruptive behavior inhibitory control focus & maintain attention despite distractionsfocus & maintain attention despite distractions attention control launch & complete tasks having long-term value, even if unpleasantlaunch & complete tasks having long-term value, even if unpleasant activation control Colorado State University 7

8 LASI 2014 Colorado State University

9 It is a capital mistake to theorize before one has data. Sherlock Holmes A Study in Scarlett If we have data, lets look at data. If all we have are opinions, lets go with mine. Jim Barksdale previously CEO, Netscape

Colorado State University 10 A student knocks Phil BeminusHello, Dr. Arsquare. Can you please help. Im struggling to understand this material. Dr. ArsquareOf course! I just read a study. The sample was typical freshman at CSU. The treatment was quite novel! What they did was special software to the millisecond Phil BeminusUhh Im in 3 rd year, sir, and a mature student. Im told my IQ is er, high. The textbook doesnt have blinking stars that prompt me to do what self explain?? My phone timer works only to 1 second. Dr. Arsquare, is that study really helpful to me?

When I Learn Online, I Dont Participate in a Well-Designed, Highly Controlled Experiment 11 FactorExperiments vs.Online Learning contentlimited crafted disconnected potentially vast wild linked nuisance variablescontrolledhaphazard treatmentclearly structured unvarying largely absent irregular learning episodesbrief review once or not at all longer free ranging review significance of contentnil or trivialself chosen & relevant consequences for menil or trivialit depends Colorado State University

12 a study reports an effect size r =.30 or d =.63 ( 74 th %-ile ) & the outcome variable reliability is r xx =.70 what effect is predicted in a replication with the same sample? we can not know exactlywe can not know exactly under a plausible assumption that.10 .40, a replication interval ranges: r lower = -.33 or d lower = -.70 ( 24 th %-ile ) r upper =.65 or d upper = 1.71 ( 96 th %-ile )under a plausible assumption that.10 .40, a replication interval ranges: r lower = -.33 or d lower = -.70 ( 24 th %-ile ) r upper =.65 or d upper = 1.71 ( 96 th %-ile ) the replication interval widens if a new random sample is drawn & the treatment is not replicated preciselythe replication interval widens if a new random sample is drawn & the treatment is not replicated precisely Im a new, non-random N = me sample. Im not likely to perfectly replicate the treatment condition(s)Im a new, non-random N = me sample. Im not likely to perfectly replicate the treatment condition(s) adapted from: Stanley, D. J., & Spence, J. R. (2014). Expectations for replications: Are yours realistic? Perspectives on Psychological Science, 9(3), 305318. thats pretty WIDE!

effects on average apply to no one in particulareffects on average apply to no one in particular and subgroups modulate the effectand subgroups modulate the effect as well, predicting the effect for an average learner is impreciseas well, predicting the effect for an average learner is imprecise if a studys treatment boosts the group average to the 74 th %-ile (effect size = 0.65)if a studys treatment boosts the group average to the 74 th %-ile (effect size = 0.65) the average in a replication will fall somewhere between the 24 th %-ile & the 96 th %-ilethe average in a replication will fall somewhere between the 24 th %-ile & the 96 th %-ile and the more I differ from average, the worse is predictive accuracy about meand the more I differ from average, the worse is predictive accuracy about me 13 Colorado State University

14 when N = me 1.do I belong to the population(subgroup) in a study? 2.how wide is the replication interval? 3.authors of websites & pdf articles dont embed theoretically coherent & empirically grounded interventions 4.variables controlled in research are not controlled for me 5.Im not average 6.I have a jagged profile of many moderator variables, not just the few named as defining the population. Im not a random representative of that group 7.you can bet the farm Ill review content before the test 8.my achievement measure isnt like the one used in a study

Colorado State University 15 arguments are powerful and best when they are knockdown, arguments force you to a conclusion, if you believe the premises you have to or must believe the conclusion, some arguments do not carry much punch, and so forth. A philosophical argument is an attempt to get someone to believe something, whether he wants to believe it or not Why are philosophers [and learning scientists?] intent on forcing others to believe in things? Is that a nice way to behave toward someone? Nozick, R. (1981). Philosophical Explanations. Cambridge, MA: Belknap Press. (pp. 4-5)

A Program of Personal Research 16 Colorado State University

17 Woolfolk, A. E., Winne, P. H., & Perry, N. E. (2015). Educational psychology (6th Canadian ed.).

applying methods & principles of scientific inquiry to reasoning or problem-solving situationsapplying methods & principles of scientific inquiry to reasoning or problem-solving situations involves skills in involves skills in generating, testing and revising theoriesgenerating, testing and revising theories reflecting on the process of knowledge acquisition and changereflecting on the process of knowledge acquisition and change Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental Review, 27, 172223. Colorado State University 18

design experimentsdesign experiments evaluate evidenceevaluate evidence make inferences in the service of forming & revising theoriesmake inferences in the service of forming & revising theories reflect on processes used in acquiring, adjusting & replacing knowledgereflect on processes used in acquiring, adjusting & replacing knowledge Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental Review, 27, 172223. Colorado State University 19

augmenting utility in AEIOU investigation skills and relevant domain knowledge bootstrap one another, such that there is an interdependent relationship that underlies the development of scientific thinking. (p. 173) Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental Review, 27, 172223. Colorado State University 20

When the focus of students experiments is becoming a better learner, they also need formative feedback (Hattie & Temperly, 2007; Shute, 2008) and particularly, they need process feedback that describes variables they manipulated to change how they learn (Butler & Winne, 1995). Winne, P. H. (2010). Bootstrapping learners self-regulated learning. Psychological Test and Assessment Modeling, 52, 472-490. Colorado State University 21

complex tasks afford genuine challenge = opportunity encouragements to experiment permission to err data what factors bear on my task? operations do I apply to information? products result? do products meet standards? what is my scientific community? professional learning science peers me, too! Colorado State University 22

Tools for Learning Online 23 Colorado State University

24 WEEE Man a 3.3-ton sculpture illustrating waste electrical & electronic equipment (WEEE) that an average British household throws away in a lifetime Eden Project, St. Austell UK are we wasting information that could support learners as they study online?support learners as they study online? advance learning science?advance learning science?

1.a browser extension for Firefox (soon Chrome) 2.very fine-grained data is logged server side 3.my curriculum = anything formatted as.html or.pdf the entire web is my librarythe entire web is my library an instructor / researcher can an instructor / researcher can bookmark key sites create custom learning materials 4.instructors & researchers can configure nStudys toolset for learning information to be studiedinformation to be studied relationships (hyperlinks)relationships (hyperlinks) tools for operating on information = how I studytools for operating on information = how I study 25 Colorado State University

26 Colorado State University

Examples of Traces & What They Model 30 LASI 2014 Colorado State University

31 as a self-regulating learner, I am a learning scientistas a self-regulating learner, I am a learning scientist help me design & carry out a personalized, progressive program of research experimenting with how I studyhelp me design & carry out a personalized, progressive program of research experimenting with how I study help me engage in productive SRLhelp me engage in productive SRL gather big data about me (and my peers)gather big data about me (and my peers) make producing data practically effortlessmake producing data practically effortless feed me learning analytics help me track feed me learning analytics help me track information I select to studyinformation I select to study operations I carry out to studyoperations I carry out to study what I learn about contentwhat I learn about content what works / doesnt work / is better for mewhat works / doesnt work / is better for me

Colorado State University 32 definescollectsanalyzes & reports data about learners & learning contexts to: understand learning optimize learning & improve learning environments adapted from http://www.solaresearch.org/mission/about/ Gitelman, L., (2013). "Raw Data is an oxymoron. Cambridge, MA: MIT Press. raw data is an oxymoron

33 Colorado State University trace = an observation of theoretical value that supports a strong inference about the operation I apply &the operation I apply & the information I operate onthe information I operate on because Im motivated to generate a product that operation yieldsbecause Im motivated to generate a product that operation yields learning analytics: definescollectsanalyzes & reports data about learners & learning contexts to:understand learning optimize learning & improve learning environment

34 Colorado State University quotes (highlight + copy to nStudy workspace) = 1.metacognitively monitor 2.plan to review quote & annotate using a note template = 1.metacognitively monitor for match to a schema 2.assemble source information using a schema copy & paste = 1.monitor knowledge 2.assemble information copied with information at the destination of the paste

35 Colorado State University # tag 1.metacognitively monitor my standard = # tag title 2.assemble information into a category = # tag title 3.plan to search for information by # tag title select a note template 1.metacognitively monitor standards = title of a schema, slots in the schema 2.plan to assemble information per the schema 3.rehearse the schema

36 Colorado State University review an item 1.metacognitively monitor recall is deficient 2.plan to assemble information (pending later event) file an item in a folder 1.rehearse (at least some of) the item to be included 2.assemble an item (bookmark, quote, chat, note, term, document) into a category represented by the title of the folder

Gathering Big Very Fine-Grained Time-Stamped Data about How I Study in the Wild Internet 37 Colorado State University

38 TacticDescription Elaborative interrogationGenerating an explanation for why an explicit fact or concept is true Self-explanation Explaining how new information relates to known information, or explaining steps taken during problem solving SummarizationWriting summaries (of various lengths) of to-be-learned texts HighlightingMarking important portions of to-be-learned materials while reading Keyword mnemonicUsing keywords and mental imagery to associate verbal materials Imagery for textAttempting to form mental images of text while reading or listening RereadingRestudying text material again after an initial reading Practice testingSelf-testing or taking practice tests over to-be-learned material Distributed practiceA schedule of practice that spreads out study activities over time Interleaved practice A practice schedule that mixes different kinds of problems, or a study schedule that mixes different kinds of material, within a single session Dunlosky et al. (2013). Improving students learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14, 4-58.

a study event = me operating on information (e.g., quoting) we can count events: How active am I?count events: How active am I? examine patterns of events: How do I study?examine patterns of events: How do I study? conditional probabilities: If I do A, what is the probability I do B next? time-structured graphs 39 a sequence of traced events: A B D B C E D B C E D A C E D A B C F Colorado State University

we can analyze content I viewedI viewed bookmarks chats I reviewed I generated, my I generated, my notes contributions to chats essays I operated on, my I operated on, my quotes #tags I searched forI searched for 41 Colorado State University

we can analyze structural properties of key terms using a defined in-terms-of relationshipdefined in-terms-of relationship see also relationship when terms co-occur in a see also relationship when terms co-occur in a sentence paragraph site 42 we can automate analyses of notes, essays & chats rhetorical building blocks: debate, explanation, examplerhetorical building blocks: debate, explanation, example content (quotes) included from primary sourcescontent (quotes) included from primary sources content generated from primary sources (notes)content generated from primary sources (notes) Colorado State University

What new terms did I learn today? list terms first seen today that appear in the text of notes How well do I understand [term X]? generate retrieval practice items relating terms in a neighborhood What should I review? show quotes containing terms not used or reviewed in N days created (N+k) days ago Colorado State University 43

we can examine social networks investigate how groups use resourcesinvestigate how groups use resources align peers to optimize collaboration in group projectsalign peers to optimize collaboration in group projects 44 who chats with whom who uses which resources Colorado State University

Robust Learners Need Big Data 46 1.robust learning = knowledge + skill + motivation each is learnedeach is learned 2.classical experimental findings offer little help when N = me 3.nStudy gathers very fine-grained data about how I study 4.big data will contain a cluster of learners very much like me highly similar peers offer a better fulcrum to leverage research for N = me than a large, diverse random samplehighly similar peers offer a better fulcrum to leverage research for N = me than a large, diverse random sample 5.big data offer raw material for wide-ranging learning analytics about how every learner self-regulates learning 6.trace data help link each learners study activities to learning science 7.big data may help spawn new hypotheses about learning & motivation Colorado State University

47 1.build systems like nStudy gather big data about studying in action my studying over time & contextsmy studying over time & contexts ditto for my community of 10 5 10 3 peersditto for my community of 10 5 10 3 peers 2.use big data to model each learners knowledgeknowledge study tacticsstudy tactics motivation: AEIOUmotivation: AEIOU 3.partition big data cluster peers post hoc, then explore: what might I need to know?what might I need to know? how might I study differently?how might I study differently? how does change affect AEIOUs?how does change affect AEIOUs? 4.feed us analytics we can use to experiment with our learning SRL present analytics in terms of how I study with nStudypresent analytics in terms of how I study with nStudy teach us just enough theory language so we can theorize for ourselvesteach us just enough theory language so we can theorize for ourselves 5.continuously update models & clusters as we study 6.help each of us track the trajectory of our research program monitor our motivation, nudge us alongmonitor our motivation, nudge us along

48 contact me [email protected] Colorado State University

Documents

Phil Winne Simon Fraser University. 1.what is robust learning? 2.what does learning science offer? 3.self-regulated learning (SRL) 4.tools for researching