Canvas Issues for the Conversion of Moodle Quiz Modules 1Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

Created on Jan 10, 2016 5:23 PM by Tim Payer 

So much of teaching Statistics is about prompting the student to express their findings with notation and numeric values paired together.This means a mixture of two data types: Perhaps a multiple choice pull down of possible notation answers paired with a numerical value. Canvas prevents this for several reasons:1.)  Text and numeric data can not appear together in a grouped question. Okay, how about if we treat the numeric value as "text"? That is a numeric "word" spelled with integers? 2.)  Okay this is a good work around. But then there is another problem: Unless the student gets all the grouped problems correct or all of them wrong the feedback given to the student will be "Partial". The "Partial" indicator is all that Canvas is prepared to give in terms of feedback to the student. The "Partial" result will only sum the total number of correct points out of the total that is possible.  "7 out of 12" would mean 7 out of 12 points have been awarded. But there will be no indication of which problems are correct and and incorrect. Flag correct answer in green and incorrect answers in red so that the student can focus on their specific mistakes, rather than remain bewildered as to which entry was made in error. And another thing. If you can ask our school to pay Canvas with six figures after a dollar sign, how come our instructors can’t ask Canvas for a numerical tolerance that exceeds four figures after a decimal point? Tim Payer

Renee Carney   Jan 12, 2016 9:16 AM

Thank you so much for taking the time to visit the Community and submit a feature idea.  Can you help us narrow and focus your idea?  We're having a little bit of trouble deciphering the issue you are seeking to resolve.  There are great tips for constructing a feature idea, including use cases and screenshots, in How do I create a new feature idea?.  Thank you.

Tim Payer Jan 12, 2016 9:37 AM (in response to Renee Carney)

Canvas Issues for the Conversion of Moodle Quiz Modules 2Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

Hi Renee, Could I send you a pdf attachment that spells the issue out in excruciating detail with screen captures and comparative displays with Moodle vs Canvas? There are several issue but let’s focus on the primary one: Grouped questions cannot give the student specific feedback. This venue of "Like" and text reply does not permit attachments. Who can I send this attachment to (8 pages) that would actually look at it? Thanks, Tim

Scott Dennis   Jan 12, 2016 10:14 AM (in response to Tim Payer)

Hi Tim,

If you would like to email me your document directly, I'd be happy to pass it along to the appropriate product manager.

Thanks,

Scottd at Instructure

Renee Carney   Jan 12, 2016 10:24 AM (in response to Tim Payer)

We are all in this together, and we are very sincere in reading all feature ideas and helping them move forward.  There is no need to be rude in our communications.

You can send your document to Scott (above), and/or me (rcarney@instructure.com).  I'm not sure that we need to look at the Moodle vs Canvas, yet.  It is a fact that different LMS features function differently.  Let's explore how the feature in Canvas is and is not functioning as you expect, first.

Thank you for specifying the focus of your issue: Grouped questions can not give the student specific feedback.

I am assuming that you are creating a quiz, then within that quiz you create a question group?  Within the question group you create questions?  Each question allows for specific feedback.  Are you seeking a feedback option that provides text to a student based on their score for the group only?

Hi Renee, (Tim Payer response at 9:33 am, Tuesday, January 12th, 2016)

Canvas Issues for the Conversion of Moodle Quiz Modules 3Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

I am Sorry Renee, I should not have been rude.

I am frustrated at the prospect of losing a comprehensive feature offered in Moodle that apparently Canvas does not offer.

I should not have been so snippy with you, Sorry.

Could I send you the attachment to better explain the issue I am having?

I appreciate your willingness to converse despite my rude comment.

Tim

(Renee Carney response at 10:40 am, Tuesday, January 12th, 2016)

Thank you so much for the apology.  That means a lot!Yes, I am going to look at your PDF now and see what we can do to help you out!

(Renee Carney response at 11:42 am, Tuesday, January 12th, 2016)

Thank you for sending your document.  We're looking at it now!

Good Afternoon Tim. (Renee Carney response at 12:59 pm, Tuesday, January 12th, 2016)

I spent some time with some of our Canvas Coaches looking over your document.  We worked to see how we could help address the hurdles you were running into.

First, thank you for the doc.  It was very well thought out.  I could tell you put a lot of time and thought into it.  I could feel your pain whenever I came to a point in your process where you

found a hurdle.  

One of our Canvas Coaches (Kona) is a Canvas Admin and teaches Statistics.  She said she uses very similar question types.  She took a screenshot that demonstrates where to input custom feedback per answer, but she did say their team has opted not to go that extra step because of the amount of time for setup (1300 question bank).

Here is the screenshot she shared.

Canvas Issues for the Conversion of Moodle Quiz Modules 4Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

Here is an example of what the student would see upon the quiz review.  (Review does have to be open for the student to see the feedback)

I think those two screenshots may help with one of the issues you were running into?

There are 4 other hurdles that we do not have a solution for, yet.  (Two of which I found feature ideas for.  I thought I had seen the others before.  If I cannot find them, these may be the 2 we want to submit)

Differentiated points for multiple answer questions Mathematical answer variations:  0.20 = 0.2  Fill in the blank questions allowing mathematical answers

oAllow Content editing for the fill in the blank A retake that allows students to correct only their mistakes

oQuiz retake work on only incorrect response

I think a few may be covered in a new Quizzes.2 project that is just kicking off.  I have sent your document on to our Quizzes Product Manager and he has let me know that he will take a look.

I will add a summary of this email to the idea thread, in case anyone is following it.  Would it be ok with you if I also shared your document there, or do you prefer it to stay private?

I look forward to hearing back from you and I'll send you more info when I hear back from the Product Manager.

Hi Renee, (Tim Payer response at 2:19pm, Tuesday, January 12th, 2016)

Canvas Issues for the Conversion of Moodle Quiz Modules 5Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

Thanks for responding so quickly. I appreciate the effort and detail you went through to explain this response.

Yes please do show the document to any and all that are interested. I actually have many more documents that I am currently sharing with our Campus LMS group.  I will send more if they are desired. But the one I sent you was the key issue.

But I think this key issue of feedback is still lacking.

When you mention in the last screen shot that "Review does have to be open for the student to see the feedback".By "Review" you mean that the instructor must check "Let Student See Answer" for a given quiz? Which is problematic: The student wants to make more attempts on the quiz, and the only way that the specific green and red flag feedback can be given is if the instructor lets the student "see" ALL the answers to the quiz the student is taking?

The written feedback notes are a nice feature, and this I have seen, however the core problem is not resolved.

Could I try and specify the issue one more time?

Ideally what I would like is for those red and green flags to pop up with every student submission within a grouped question set.But it is important here that the correct answers are NOT revealed to the student. With unlimited attempts the student can then know which problems are wrong and then work on correcting those wrong answers on subsequent attempts. If the answers are given (as in Review Mode) I am sure that you are aware that many students will simply copy the correct answer on their future attempts. (The written comments are nice but secondary.)

My current understanding of how Canvas runs quiz feedback now is as follows:

1.)  Grouped questions must use all the same question type:      Either all "Multiple Fill in the Blank", or all  "Multiple Drop-down".       It is not possible to mix data types of numeric and text.      It is not possible to weight individual cells within a grouped question with different point values.

2.)  "Partial" is the only feedback that a student is permitted for a grouped question. The student will be given a point tally of corrected points submitted as in: "7 of 12". But the student will not know which her entries are correct and which are incorrect. (This will frustrate the student, and this is the key issue.)

3.)  If each and every cell of a grouped question is separated into individual questions then an instructor will have control of the data type used and the points assigned. A student will have their incorrect answers red flagged, and correct answers will not have a green flag. The blank non-response is the indication of a correct answer. (A green flag would be nice, but again this is secondary.)

So it seems there is no way around Issue number 2. Does this seem to be the case?I can work around all the other issues but that "Partial" feedback issue is going to be a major problem.

Please let me know if you can think of any other work-around solutions.

I appreciate your efforts Renee.Sincerely,    Tim

Tim! (Renee Carney response at 4:18pm, Tuesday, January 12th, 2016)

Canvas Issues for the Conversion of Moodle Quiz Modules 6Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

We'll keep trying until we get this 

Thank you for the permission to share the document.

There is one small setting that I think may help you when it comes to the review.  You can allow students to see if their answers were correct or incorrect, without giving the correct answers!  I took a quick screenshot for you.

"Let Students See The Correct Answers" should be deselected if you want the correct answers hidden.  Does this help with the main issue?

Responses to your numbered questions/issues:

1) You are correct that the multiple answer questions (or as Moodle calls them, grouped questions) are either dropdown select, or text.  There is no mixing and matching.  You are also correct that answers cannot be weighted within those question types.  I think these two ideas would make perfect feature idea submissions!

2) If you allow the student to review their quiz, but turn off the correct answers feature, they will be able to see which answers were incorrect without giving them the answers!

3) You taught me something new today!  I guess I never realized that the correct answers did not have a flag.

Below is a screenshot that demonstrates 1) what the review looks like with correct answers turned off.  2) using

the feedback tool to tell the student when a multiple answer - answer is correct.  (don't mind the cows,   when you build a test course, you use what you know!)

Canvas Issues for the Conversion of Moodle Quiz Modules 7Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

Let me know if we're getting closer!

Hi Renee, (Tim Payer response at 4:18pm, Tuesday, January 12th, 2016)

I appreciate all the detailed screen shots as it helps communicate the point, but Sorry no, we are not any closer.

Many students will be making multiple submissions (between 4 and 50 submissions per student) on a quiz module that has been adapted for homework assignment submissions.

We want the student to check their notes, talk to class mates, come in to office hours, struggle and search the web in an effort to find the correct process for a calculation or a text answer before the due date. Revealing the answers to the student would only make sense After the assignment is due. Otherwise the student will take the correct answer and submit it in their next entry.

So I am afraid we are back to square one on the main point of specific feedback on grouped questions.

Thanks for the persistence though,  Tim

Canvas Issues for the Conversion of Moodle Quiz Modules 8Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

P.S.  Cows are sweet. I live next to a pasture for grazing cows and they sometimes wander up to the barbed wire fence that separates our back porch and the pasture just five feet away from our porch couch. If you remain perfectly still they wont spook.

Hi Again Renee, (Tim Payer response at 4:38pm, Tuesday, January 12th, 2016)

Let me take another look at your cow problem. I misunderstood what you were implying with the "correct answer" response.

You are saying that correct submissions within a multiple answer problem will be flagged with a "correct comment" and the incorrect replies will not be flagged with a response?

This helps, yes, I think.

So lets say I had a multiple answer question with 9 cells for the student to enter. If the student made 5 correct entries of those 9 cells, that student would see 5 "correct comments" next to the summary of their submitted answers? And no comment will be next to their incorrect answers letting the student know that these submissions are in error? Provided of course that I have created such a comment for each submission?

I am going to give it a try on some quiz assignments that I had separated into single question single responses.

It will take me a while to do and test but I will get back to you tomorrow.

Thanks so much for your persistence,

Sincerely,  Tim

(Renee Carney response at 6:55am, Wednesday, January 13th, 2016)

The grazing cow pasture next to you sounds amazing!  We raise a few cattle.  Ours are pretty tame and love to come to the fence for a treat and a good scratch!

In your last email, it looks like you are on the right track for testing.  Looking forward to hearing how it works.

Hi Renee, (Tim Payer response at 8:48am, Wednesday, January 13th, 2016)

Okay, I have made a test run with my first Homework problem, and I think that the solution you offered of letting comments for correct feedback function as a flag will work! Or at least we are closer.

Thank you for going the extra mile to offer a solution.

It is going to be a lot of extra work though. Numeric values in particular when entered in the "Multiple Fill in the Blank" mode will require the creation of many possible "correct" answers.

For example if a correct answer is numeric :  "0.1234" this would have to be considered as text with a particular numeric spelling. And if I want to give a tolerance of plus or minus 0.0005, this would mean writing up 21 possible numeric words of accepted spellings:

For example: The following would all need their own comment to be able to be flagged as correct:

0.12290.1230

Canvas Issues for the Conversion of Moodle Quiz Modules 9Stat 109 Trial, Spring Semester 2016 Issue #1a Email exchanges

0.12310.12320.12330.12340.12350.12360.12370.12380.1239

And another 11 "spellings" for students who clip the leading zero:

.123

.1229

.1230

.1231

.1232

.1233

.1234

.1235

.1236

.1237

.1238

.1239

It will mean at least 4000 to 5000 entries of the comment of "Correct!" for 30 plus assignments, but this is preferable to the lack of feedback for the students.

I better get to it.   

Thanks for your help,

Tim