Transcript - 5 Awesome tools to improve survey results...Transcript - 5 Awesome tools to improve survey results Dan Fleetwood: If you have if you move all your answer options over

Transcript - 5 Awesome tools to improve survey results

Crystal Wiese: I only have states left in the world to visit or in the United States to visit and South Dakota and North Dakota on there, but I don't know what I'm going to get up there. Um, okay. So one more minute. We're going to get started. Crystal Wiese: It's great. Crystal Wiese: Not able to hear anything. Crystal Wiese: Huh. Crystal Wiese: This is, Am I audible. Can everybody hear me all right, Dan, you want to do a little quick test. Dan Fleetwood: Yeah, I can hear you. Crystal Wiese: Great. Okay. Crystal Wiese: Perfect. So sorry, Andre maybe check your Dan Fleetwood: So he can or they can hear now.


Crystal Wiese: Okay, perfect. Um, okay. So I think it's about time to get started. Feel like five minutes is good cushion time and thank you everybody for coming. We have Dan Fleetwood here to talk about five ways to improve your survey results and we are stoked to Crystal Wiese: Really help you guys. Um, oh, you know, jumps job stuff into the Q and A or into the chat window. Crystal Wiese: I'm Crystal Wiese: Closed captions. I don't know. Great question. Julie, let me I will work on that I'm Crystal Wiese: Drop yeah into the Q and. A or into the chat will be gathering them and we'll be answering some of them after Dan is done and then Crystal Wiese: We'll be sending out an email tomorrow or later today with a link to the video or the presentation, along with the Q and A's so that you can have them in your email box and you can listen back to them. So without any further ado, Dan get started. Dan Fleetwood: All right, we'll do hey thanks crystal appreciate that everybody excited to join you today and talk about five ways to improve your survey results. Dan Fleetwood: As Crystal mentioned, I'm Dan Fleetwood I head up the research and insights platform here at QuestionPro so that platform consists of the research edition. Dan Fleetwood: License. The, the surveys with the max diff discrete choice conjoint heat map hotspot all of those advanced question types, along with Dan Fleetwood: Some more advanced features, some of which will cover here today also includes the communities platform.


Dan Fleetwood: And then our audience platform as well. And then these services and partnerships that we might use to help our clients get the get their projects done Dan Fleetwood: So that's the research platform that is what I lead here at QuestionPro. But today, let's talk about these five tips and we'll go over them. Dan Fleetwood: And somewhat in depth, I would say that will give you, I'll give you definitely a good understanding of all of these different features. Dan Fleetwood: Not too in depth, but enough to make it dangerous, especially on the platform and hopefully spark some interest on different things that you can do. Dan Fleetwood: To really help out your survey results. So today will cover how you can create unbiased or reduce the bias in your surveys using randomization. Dan Fleetwood: Talk about customizing surveys based on answer up previous answer options and questions. So really about looping Dan Fleetwood: Setup team testing options and our collaboration mode advanced analytics talk about weighting and balancing Dan Fleetwood: And then also ways that you can improve your survey results when the respondents taking the survey and around some of the data quality tools that we have Dan Fleetwood: So without further ado, let's get into it. Let's talk about randomization, so obviously randomization. You can randomize answer options. You can also randomize questions.


Dan Fleetwood: In the survey in order to reduce bias and improve results. This can also be used for things like concept testing image testing even Dan Fleetwood: Decrease survey fatigue, because if you have a long survey, you can route people through different channels, thus reducing the length and improving the overall Dan Fleetwood: You know, time that it takes to complete the survey, but really randomization is used for randomizing answer options or questions to do spice, or to implement different testing inside of a survey. Dan Fleetwood: So the, the four types of randomization that I want to cover today basic answer option randomization advanced answer option randomization. Dan Fleetwood: Question randomization and then block randomization. I'll show you where each of these are inside of the QuestionPro platform and then talk a little bit more about the more in depth as well. Dan Fleetwood: So basic randomization. Here you can find it under Settings and then Dan Fleetwood: Enter display options random and this essentially does what it says right there randomized is the answer options so that they're not displayed in the same way. Dan Fleetwood: To each user obviously we've sure you've all used this inside of the platform on other survey platforms as well. The more you know kind of fun stuff is the advanced answer after answer option randomization, where you can do things like Dan Fleetwood: Randomly display all items from the list below you can choose to randomly display a certain number of items from the list so


Dan Fleetwood: If you have if you move all your answer options over here and let's say it's a long list and you want to randomly display five of those options. Dan Fleetwood: You can do that here five of those options will display so it can help you know reduce the number of options that you're asking to each person in the survey. Dan Fleetwood: And then you can choose not to display items from the list below as well. So these are a little bit more advanced option randomization, especially when it comes to Dan Fleetwood: The only displaying, you know, a certain number from the actual list. Now when things get interesting. You can use question randomization, you can find that here. But once you enable the blocks. Dan Fleetwood: Once you have different blocks in the blocks of different questions and then you can randomize the questions within those blocks. So, um, you can find that here under question randomization at the block level. Dan Fleetwood: Once you do that, you can click on it and you get these different options so similar to advanced randomizing answer options you can do these on a question as well so you can randomly display all the items from Dan Fleetwood: The list. So if I wanted to randomly display. Like, what is your gender and estimated household income, I can move those over here in this area and it would randomly display. Dan Fleetwood: You know those one of those questions. I can also choose to randomly display. Dan Fleetwood: A certain number of items from the list below. So if I wanted to randomly display one of these questions to the users, I can do that.


Dan Fleetwood: This also works well and major concept testing. Let's say I have four images I want to randomly display those two a user or a respondent rather that comes in. Dan Fleetwood: I can easily do that here just by clicking on these dragging them over and then display in those results. So really making it easy to do. Question randomization and in addition Dan Fleetwood: More advanced randomization. See Dan Fleetwood: It says loading, for me, is that what you guys see as well crystal or mark. See Dan Fleetwood: Here, I'll try this. Okay. Crystal Wiese: Sorry, I was muted. Crystal Wiese: Um, I saw it when you XLed out Dan Fleetwood: Yeah, it says loading here. I'm here. I'll just Dan Fleetwood: I'll go with this view. Dan Fleetwood: I'm not sure why but


Dan Fleetwood: Hopefully you guys can see that Crystal Wiese: Okay, I can do that again. Okay. Dan Fleetwood: Maybe it's just that one slide. Okay. We'll just move past that slide. All right. Dan Fleetwood: So block randomization, essentially you you go into there's a tools. I was just going to show you how to get here. Dan Fleetwood: There's a tools menu item you click the tools and and there's block randomization or block randomizer and you can get that go so you can get that Dan Fleetwood: Block randomization by go in there. So essentially what this allows you to do is randomize blocks of questions. So Dan Fleetwood: There's you can drag and drop the different blocks inside of the randomizer and then choose to randomly display those different blocks of questions. So again, could reduce bias. You could also do this to Dan Fleetwood: Implement different types of concept or different types of tests inside of your survey really make it smarter overall with some of these advanced block randomization and Dan Fleetwood: Logic options here. So really making it easy. You can choose to randomly display these items. You can also add quarters to it. So if you wanted to. Dan Fleetwood: You know, make sure that you have even distribution, you can do that as well. So these are all different randomization options inside of the, the question for research platform.


Dan Fleetwood: So looping let's move on to looping and cover this so I'm sure we've all either use a little being heard of it. Maybe we've seen it as we've been you know clicked around or created service. Dan Fleetwood: Inside of the QuestionPro platform. So really it's when a set of questions is asked multiple times for different answer options that were selected in a previous question, so Dan Fleetwood: You might have a multi select question and then based on those answer options that they responded selected. You want to then ask some follow up question. So at a very basic level, we can go over this example. Dan Fleetwood: Please select all of the following pizza restaurants that you have ordered from the last six months. So there's Pizza Hut Domino's Papa John's us pizza and then the follow up question is how would you rate pizza from Dan Fleetwood: The restaurant and this can be for based on Dan Fleetwood: What the respondents selected right so if they selected. Let's say pizza hut and dominos, it will rotate this question through for both those answer options. If you have more questions like, not only how would you rate the pizza. Dan Fleetwood: Maybe you also have, how would you relate the delivery, I would you rate the delivery experience with order experience. You can have any number of questions and have a look through for each of the answer options selected here. Dan Fleetwood: So how can you enable looping inside of question Pro. So you've probably seen it under logic here and then the next step is Dan Fleetwood: Going to skip logic and then enabling this looping option here and then


Dan Fleetwood: If you can, you can have a jump are selected. So a jumps to the the follow up question that you want to ask us. The first one is fine. Dan Fleetwood: And then input the piping text and then save your logic and then you're good to go there. So, making it really easy for you to enable looping inside of your survey and Dan Fleetwood: Really add this piece to reduce survey length and also reduced programming time. Honestly, you don't have to add a bunch of questions with show hide to answer options. Dan Fleetwood: And things like that so it could really reduce the amount of time that you're spending and programming surveys. So looping about simple to implement and it's really a powerful tool that you can use. Want to help out with overall survey flow and also to reduce programming time as well. Dan Fleetwood: So I want to talk a little bit about collaboration mo. So, Crystal Wiese: Question. Crystal Wiese: I can answer a little bit Crystal Wiese: Can you use skip logic within looping Dan Fleetwood: Can you just skip logic within looping so you can. What you can do is you can base a loop off later on in a survey so it doesn't have to be directly. The next question we've enabled it so you can essentially do delayed looping now. So that's something that you can do.


Dan Fleetwood: Skip logic within a loop. I think it would depend on the exact use case. So I'd be happy to follow up with whoever asked that question in a specific use case. Dan Fleetwood: After the webinar today. But essentially, there's different options that you can do with looping to make the survey smart with with logic and so forth. So kind of depends on the exact use case. But I can definitely get back with the person that asked the question. Crystal Wiese: Okay, awesome. All Dan Fleetwood: Right. And then so collaboration. So once a survey has been programmed and you want to get feedback or input around the survey. Dan Fleetwood: You can enable collaboration mode and then you can send via email a link that that person can use to join a version of the survey and they can Dan Fleetwood: Then comment on the different questions. Add in their thoughts and so forth feedback that they want to give based on that particular Dan Fleetwood: Survey overall or the question. So once you enable collaboration, you can enable the status here so you can collaborate and then Dan Fleetwood: You can choose if it's a distribution list or if it's everybody inside of the organization. You can choose individual collaborators, like Dan Fleetwood: You know, I put my email address in here. And then I hit send an invite. I get a link and that takes me to a collaborators view like this where I can add comments all over the entire survey. So, you know, love the survey, just make the changes and will be good. You can see


Dan Fleetwood: The timestamp of when it was added, who added it. You can also do this on a question level as well so you can suggest changes. You can also add comments, much like when you're using, you know, different collaboration features inside of other Dan Fleetwood: You know, like Google Docs or or Microsoft Word or whatever it might be. You can have these comments and and then people can go and then Dan Fleetwood: Review those comments and she's doing either enable them or you know approve those, those, those comments or whatever they might want to do. You can see that you get a log of each one here. So really helps enable Dan Fleetwood: I think work to get done, especially now obviously when more and more people are still working remotely and Dan Fleetwood: Maybe you can't go to that person and have them, you know, pull them over and have them review the survey, some of that getting that feedback and inside might be a little bit more challenging but Dan Fleetwood: With the collaboration view we hope to overcome some of that and being able to provide feedback. Dan Fleetwood: No matter you know where you are or also outside of your organization to. So maybe you want to share it with external partners or clients, you can share that link and get their feedback that way as well. Dan Fleetwood: All right, weighting and balancing. So this is a newly revised module here for us at QuestionPro we've added in some some features. But the kind of give a baseline. Obviously, there's many different algorithms formulas techniques thoughts opinions on waiting and balancing but


Dan Fleetwood: Just to establish a baseline, you know, weighting and balancing is a statistical methods used by researchers Dan Fleetwood: To get rid of bias and the survey data to reflect the variables of a general larger population. So permitted to change the weights of variable fields. Dan Fleetwood: And avoid sample bias. So if you're not familiar with weighting, really. Let's say you are a men's clothing company. Dan Fleetwood: And you know that like % of your, of your customers are male it when you get the survey results back that's more of a gender split and you want to weight the data accordingly. Dan Fleetwood: To to account for that to make sure that the data that you're analyzing reflects your, your target market so Dan Fleetwood: Really it's loading again here out of this. So to find to find the find weighting. You can go to analytics manage data and select weighting. You can go to Dan Fleetwood: At once you add your question in here. Here, I'll try to present again see Crystal Wiese: Just strange bug. Dan Fleetwood: Yeah, it's not loading for some reason. Anyway, okay, it's this might be a little hard for people to see this, but I'm not sure. Crystal Wiese: You get a full view and in full screen maybe Yeah.


Dan Fleetwood: Because it was working. Anyway, so you can do you can do a balance. Weight, so a balanced way does off of a variable. So rather than Dan Fleetwood: On proportion might be a little bit hard for you guys. See, I apologize for that. But you can base it off of a percentage or you can base it off of actual variable way right so Dan Fleetwood: Two different ways that you can get the the way to data or the weight the weights inside of question Pro. You can select the question. Dan Fleetwood: And then either at it based on percentage or you can add it based on a weighted variable. And that's what's new is previously you could only do it on percentage, but now you can do it on Dan Fleetwood: The way to variable. And you can also be a base it off of multiple variables as well. So if you want to try this one more time here. Dan Fleetwood: Okay, you can base it off of gender and income here so you can base it off of two or more questions and then weight the data appropriately there. Dan Fleetwood: And then finally, another thing that you can do is you can import weight. So let's say you're working with your data externally and Dan Fleetwood: You want to you know what weights, you want to use. You can import those weights without having to go in here and add your questions. And so the ability to import weights is is a new feature as well but Dan Fleetwood: You know, waiting the data will help help you help the data over also reflects the the target market that you're you're going after if that's something that you want to do, but definitely the


Dan Fleetwood: Waiting with having the variable and being able to import the weights is something that is pretty new here at QuestionPro and it's an important feature that we got actually a lot of requests for so Dan Fleetwood: If you are familiar with weighting or you want to try it out. I would encourage you to go to, you know, do it to try it out on a data set and see how it works for you, but it's definitely Dan Fleetwood: A good feature to use to ensure that you have I quality survey results right let's try this presentation mode here. One more time. I think so. I want to talk about a little about data quality so Dan Fleetwood: We have built in data quality tools inside of QuestionPro that you may not have seen or maybe you've seen, but we're we're curious of how they actually worked and and all practicalities so Dan Fleetwood: Really, they can be used to flag responses based on settings that are enabled like jibberish open ends straight liners speeders etc. And we'll talk about all of them in detail in just a minute here. Dan Fleetwood: But it's one of the, I think it's one of the most effective ways to ensure high quality data right to make sure that Dan Fleetwood: Some safeguards are guardrails up on the other end to make sure that the person that is taking the survey. Dan Fleetwood: Is, you know, hold accountable for some of these things that they might do. And you know, when you get this data back, obviously it's not usable for you so you need to ensure that Dan Fleetwood: You're able to catch these people as you're doing it, especially because you know quarters and other things might depend on this so you can inside of question probably flag, these, these different answer options.


Dan Fleetwood: So to find the different data quality tools that we have go to analytics and manage data here and then Dan Fleetwood: And then you can some of the different data quality options that we have here are one one word answer so it'll identify one word answer options. So someone types in Dan Fleetwood: Obviously, one word in an open ended response. It'll flag that so you can go and review that at a later time. Dan Fleetwood: Additionally pattern responses. So if someone is typing in the same word over time, or there's a straight line or zigzagging Dan Fleetwood: In answer options. Those that pattern will be flagged so that you can go back and review to see if it's a legitimate response or not. Dan Fleetwood: Similarly, we have jibberish words. So an open ended responses. If someone is just typing keys at random and you want to have those interruptions flagged, that's an option as well. Dan Fleetwood: Duplicate texts across responses. So this is something that you can enable and then it will check to see if they're duplicate answers across different responses and little flag it Dan Fleetwood: And we allow the first response to pass and add a flag for subsequent responses. Another one is duplicate IP address a duplicate Dan Fleetwood: IP address across different responses. So if you want to ensure that it's not one person repeatedly taking the survey, obviously we have


Dan Fleetwood: Some safeguards in place for that, but you can also use this if you're using, you know, third party external sample. Maybe it's a list that you're not too sure of. Dan Fleetwood: These are these are, this is something that you can enable to help identify the if there's sort of the same person. The same IP address, they can the survey multiple times. Dan Fleetwood: And if all checkboxes are selected within a multiple, multiple choice question. This can be flagged as well. And then speed trap. So if someone is completed the survey. Dan Fleetwood: At a faster average or faster than average, then this can be flagged as well. So inside of the platform. This is essentially what it looks like as we add a flag to the actual response. And then you can click to review the response as well so Dan Fleetwood: Here you can see that the question, What do you like about pizza. This is the value that was inputted obviously a bunch of, you know, jibberish Dan Fleetwood: jibberish word was typed in. So that gives the reason for the flagging jibberish word. So then you can come in here and check if you want to delete the response, throw it out. Dan Fleetwood: Or you can unflag it, if, if you review it and it turns out it is a legitimate answer, obviously, this one wouldn't be. But if it's a straight line or speeder or something like that and you want to unflag the response. You can you can do that here as well. Dan Fleetwood: All right, let me present here and then I have a few bonus tips that I wanted to go over these are more practical in nature, not necessarily related to something special that QuestionPro does, but always things that you want to, you want to keep in mind when you are Dan Fleetwood: When you are conducting a survey setting it up, etc. So clearly define you know who you want taking the survey. I think this can often go missed, especially in a lot of the surveys


Dan Fleetwood: That I see where people send the survey out, they get all their answer they get all the answers collected and at the end, it turns out that they didn't actually Dan Fleetwood: Recruit the correct people to participate in the survey. And this is where I think screening to make sure you have the right audience is definitely Dan Fleetwood: Care about and make sure you have the right people, completing the survey that you want, rather than later on ending up and realizing that on a way to actually didn't. I didn't collect Dan Fleetwood: The answers from the right respondents. And I didn't get those right survey results, fundamentally, this is where, you know, you want to make sure Dan Fleetwood: That you're getting the right people in all the other stuff I talked about today is definitely good and will help identify some of that, but really having a clear screener i think is Dan Fleetwood: A good reminder for all of us creating surveys and keeping that in mind as making sure you're initially screening for the audience and that the sample source that you're getting it from as high quality. Dan Fleetwood: And matches your desired audience, right, like just taking that extra step that extra time to really review and make sure that the list, you're sending out to is legitimate. Dan Fleetwood: That it's not filled with a bunch of dummy records or things of that nature, and also ensuring that the sample. Dan Fleetwood: That you're putting through especially if it's, you know, third party sample or maybe a list, you're getting from a client. Dan Fleetwood: That it's been vetted and then also for third party sample. Make sure that the partners, you're using have those data quality tools and and good


Dan Fleetwood: safeguards in place, like we do. I know for all of our audience projects you know there's there's a ton of checks that we do to make sure that our clients are getting Dan Fleetwood: High quality data whether that is through the self service function that we have under audience inside of the platform that you've probably seen in the drop down or it's, you know, if we're Dan Fleetwood: That's not the self service and it's more of a managed service, you know, that's something that we do as well but Dan Fleetwood: I think you know clearly defining the want taking the survey, making sure you're quickly screening for them. And then, ensuring Dan Fleetwood: Sample source is of high quality as you can get. Are some definite bonus tips that I want to just to bring up to you today. And just to remind you of that. So I know there were a few or there was a question asked Dan Fleetwood: Around looping, I think there's a few more questions coming in. Yeah. Dan Fleetwood: Yeah, I'm ready to go. Crystal Wiese: Um, let's do you want to start at the top and work our way down. Sure. Crystal Wiese: Sure. Okay. Kim collaboration be done after the survey is live and active.


Dan Fleetwood: That you can't. You need to do it before, otherwise you'll change the status from active to collaboration. So you want to, you want to do it before. It's a large survey. Crystal Wiese: Okay, well, how do you get to the collaborators view. Dan Fleetwood: So the collaborators view you can get there. What you'll get sent a link and then or you can base it off of the survey itself. Actually, let me just try something. Dan Fleetwood: So here you can, sorry. Crystal Wiese: I was gonna tell everybody drop questions in because even if we don't get to them live. We will send answers out afterwards. Dan Fleetwood: Perfect. So you can get to the collaborative view here so you can search or you can edit the status, go to collaborate and then once you're inside of a survey here, let's say, Dan Fleetwood: I turn this one to collaborate. I can invite collaborators to view it or to send it to them. I send the invite. They get a link to the survey and then essentially it's it's just this particular link here that one. Dan Fleetwood: See Dan Fleetwood: It's this link here. Dan Fleetwood: And then if I click on


Dan Fleetwood: Then you can see here that you can see the general comments you can add a comment and you can invite other people as well to send maybe you want to send that specific comment to an email address. Dan Fleetwood: And here's where you can give specific feedback for the comments different email addresses that you want to Dan Fleetwood: Highlight or, you know, to kind of tag that person on. You can also suggest changes to the question type as well and update that. So number number of different things that you can do inside of the collaboration. Crystal Wiese: Great. Crystal Wiese: Doo. Let's keep going. Um. Oh, this one's kind of long from Dan who use the same name as you are you are you able to edit the survey questions directly from the cab collaborators review. Dan Fleetwood: Or can Dan Fleetwood: You see can suggest that it's and then you can go in and then essentially approve those. Okay. That's great. Crystal Wiese: And how are the answer to the survey logged in collaboration mode. Dan Fleetwood: So, the answers are still logged as normal and then you can view it as a test response. But what you really get is


Dan Fleetwood: You know those particular comments here you can see the log here, and then you can go and approve these if you're the admin, you can do that. And then you can see you can still see the response as you normally would. When when testing or when looking at those responses. Well, Crystal Wiese: Okay, now we have some waiting questions. Dan Fleetwood: Mm hmm. Crystal Wiese: Is waiting get export it into the SP S. S. Dan Fleetwood: Oh, I see that from Ben. Yeah. So Ben, this is actually Dan Fleetwood: I guess phase two of the waiting and balancing is exporting into SPSS, and I believe it's slated to come out in June or July I can follow up with you on that for sure, but a step one was getting the updated waiting and then I know phase two was export into SPSS. Crystal Wiese: What are some pros and cons of waiting Dan Fleetwood: Yeah, so there's there's different schools of thought on this, and I won't go too much into it but Dan Fleetwood: So there's definitely Dan Fleetwood: People that say you should wait your data that you shouldn't wait your data. There's people that say you should only Dan Fleetwood: Wait up your data or not. Wait it down or maybe vice versa. So there's there's definitely different schools of thought and that how much you alter with the


Dan Fleetwood: Data set how many completes, do you have, do you have a large sample that it makes sense. I think if you have a small sample size. Sometimes Dan Fleetwood: It can bias that data and it's not really a true reflection. So the waiting that you do won't make a lot of sense and could really skew the results so Dan Fleetwood: Some different different schools of thought on wedding data, but I think largely depends on on your comfortability with it and then the data set that you're actually dealing with It. Crystal Wiese: And for data quality options, is it set up per survey created Yet script. Crystal Wiese: answered your own question. Crystal Wiese: What's this next one would it delete just the question response or the entire respondents question. This is when you were talking about data quality. Dan Fleetwood: Yeah, so we would you can. So if you choose to unplug. Let me just see if I can find the example I was working on earlier. All right here. So this data quality. Dan Fleetwood: If I go under analytics and then responses here so I can see you can see that this response ideas flagged and this is the one that I showed you earlier. As an example, but


Dan Fleetwood: What do you like about pizza. It's flagged as its value. So I can unplug this response. Dan Fleetwood: And then it's on flag. So it does nothing there if let's say this response was flagged and he wanted to delete it. You could delete it there. Dan Fleetwood: Or you could you could also go and, you know, bring this up and then edit the response if you need to. Just to fix something maybe was Dan Fleetwood: Look like some rubbish words but you could make it out. You could fix it there so it would it would you can delete the whole whole question or the whole response. Dan Fleetwood: That's essentially what you would do. Yeah. Crystal Wiese: Here we go. So is there a way of flagging to the participant themselves why they are filling in the open ended text box that their answer is not acceptable. Dan Fleetwood: So there are some validation options that you can do. Dan Fleetwood: In terms of if it doesn't meet a minimum length requirement or there's a few others. But in terms of like the exact that doesn't necessarily hundred percent match some of the different Dan Fleetwood: Data quality options that we have here, but there are some safeguards that you can put in place via validation check in there. Crystal Wiese: Good to know.


Crystal Wiese: Can I place a quota weights and not kick people out of the survey. But Heather survey responses flagged. Dan Fleetwood: You can, yeah, you can do that. Great. Crystal Wiese: Do you want some more. Hold on, is reducing bias. The only reason to use randomization. Dan Fleetwood: I think it depends, you know, some of the different the question randomization and block randomization. Dan Fleetwood: Those wouldn't be so much for reducing bias, although it could be that would be more for using it for different study types like different image. The main Dan Fleetwood: Some of the main use cases that I see for the question randomization and block randomization are for image testing or have these five questions or have these five Dan Fleetwood: Maybe different series of questions and I want to randomly display one set to a respondent and and so it's really those that would be more for different study types or or methodologies, I think. Dan Fleetwood: Really when it comes out of just sort of a question level and randomizing Dan Fleetwood: The main reason that I'm familiar with is just to reduce, reduce bias, but there may be others that are out there and other people have different schools of thought that that's the main reason that I see it, especially when you randomizing answer options in a particular question.


Crystal Wiese: Gotcha. Crystal Wiese: Can collaboration be used for alive survey scripting. Dan Fleetwood: You can, you can, yeah. So once the survey is live or you're sending it out. You've already sent it out and responses are live responses being collected. You wouldn't want Dan Fleetwood: To have it in collaboration mode, but before that process and you're initially testing that you can use collaboration mode. Crystal Wiese: Great. Well, um, what else do we go Crystal Wiese: Oh, this is a good one and one. I'm sorry. Crystal Wiese: I thought it was interesting. Crystal Wiese: Maybe not. Crystal Wiese: Probably have one IP for all four employees if they're all at the same place. Dan Fleetwood: Yeah, if you have that, then you wouldn't want to enable the you wouldn't want to enable duplicate IP across responses, because then it would it would flag them all. Okay. Crystal Wiese: Well, um, no. Dan Fleetwood: Other other other questions, there's a


Crystal Wiese: I think that's it. Um, we can do one. Can you show. I don't think this might take too long. Can you show us how to set up quotas and weights that flag a response, but I'm not Dan Fleetwood: I think that probably be better handled offline. So I can, we can reach out to you directly. I will reach out to you directly with that. Crystal Wiese: And I don't want to take up too much of people's time and Crystal Wiese: That's I think those are some good Q and A's, though. Dan Fleetwood: Ya know, that was really good. If you have any other questions after this, feel free to reach out to me. Crystal Wiese: And Crystal Wiese: We will be out. Yeah, we'll be putting all the questions and answers together and then sending them out with the video and the transcripts. Crystal Wiese: Oh, can you talk a little bit more about the speed or section of data quality. Dan Fleetwood: Yeah, so the speaker section here kind of looks at the average survey time and then Dan Fleetwood: It looks at the average time to complete the survey and then you can you can set the time that you want to flag if it's a


Dan Fleetwood: Seconds minutes. Hopefully it's not ours, that would be a long survey, but naturally it's there because there are those times. So you can set it. The average time to complete it, or you can base it off of Dan Fleetwood: Us real time average. So it'll, it'll calculate, obviously, it looks at the average time and then you can Dan Fleetwood: Pick a deviation allowed from that average of its you know seconds to minutes, whatever that might be. Then you can add that that deviation and then it will flag responses accordingly. There Crystal Wiese: Well, I guess if nobody has any more questions. Crystal Wiese: You know, we're going to give you back some of your time today. But we thank you so much for joining us. I'm Crystal Wiese: Dan, do you have any parting wisdom. Dan Fleetwood: No, I would say definitely reach out to me if you have questions feedback inside. I love to see here and see how people are using Dan Fleetwood: The platform. And more importantly, how they want to use it. So definitely let me know happy to answer questions and help you out or route you to someone that you know can help you. So I appreciate everybody's time today. And yeah, I think that about wraps it up. Crystal Wiese: Awesome. Thanks so much, Dan. Dan Fleetwood: All right. Thanks, everybody. Have a good day.

Documents

Transcript - 5 Awesome tools to improve survey results...Transcript - 5 Awesome tools to improve survey results Dan Fleetwood: If you have if you move all your answer options over