As a homework service, Quest is entirely capable of supporting online homework, learning modules (with lecture videos embedded), and exams remotely. More than 300,000 online exams have already been administered, so while we're continuing to add functionality, you can be confident the basic processes and software pathways are extremely well exercised.
Exams can be constructed by selecting 'learning module' as the assignment type, and then selecting 'test mode'. You can enter in a specific amount of time students will have, and if you'd like, a window in which they will have to complete the assignment. We are currently testing both commercial and within-system proctoring options and will be communicating more shortly. A short overview of these options are included below*.
While many questions in Quest already offer random number generation to offer a unique problem to solve for each student, if you'd like to introduce more variability you can choose to pool questions together to ensure that all students don’t have the same questions either.
Q: Is it possible to give an hour time limit on the exam, but the exam can be taken at any time on a particular day for a student?
A: Yes, absolutely. In order to do this set the (start date/due date during the full window you'd like students to be able to take the exam. Then enter in the time limit. Once a student proceeds off of the initial instruction slide of the learning module their timer will start; at the end of the expired time they will be prevented from accessing their exam any more (even if the window the assignment is open is still in duration). Students will not have access to the exam after the due time, so if they started late they may not finish. (For example, if you set an exam to go from 2-4pm, and a student started at 3:55pm, they would get cut off after 5 minutes of working on their exam.)
Q: Is the functionality of the start/due date times and time limit redundant?
A: It depends–if the start/due date time is the full amount of time allotted for the exam, yes it is redundant. If the start/due date times offer a wider window during which students can access the exam, the time limit will be the limiting element and the two elements are not redundant.
For example, if the
Start: 9:30 a.m.
Due: 11:00 a.m.
time limit: 90 minutes
the time limit doesn't matter, because the full window was the full time limit. If a wider window was offered to students (say, start: 9am; due: noon; time limit 90 minutes) then the time limit would matter--in such a case students could log in anytime between 9am-noon and access the exam. For those who started their exam between 9-10:30, they would have a full 90 minutes to complete it, and then be timed out. For those starting after 10:30am, they will have until noon to submit answers (we saw that they may still be 'in' the exam but all submission boxes are grayed out preventing further input). Even if someone started at 11:50am, they would stop being able to enter in answers at noon, because that is when the assignment is due.
Q: Some of my students have extended time accommodations. Can this be accounted for?
A: Yes, please just send us (at questhelp@austin.utexas.edu) the unique the student is in, the students' eids, and how much more time (1.5x, 2x, etc) each is to be granted. We will ensure the students receive this accommodation for all of your Quest assignments this semester
Q: I was planning for a bubble sheet exam, but now need to make it online. Is there a direct way to do this?
A: Yes, in just a few clicks you can get your preexisting test set up for remote administration. Navigate to your original exam and click on the export questions on the left side bar. In a separate tab create a new assignment, a learning module in test mode. Next, navigate to the left side bar in your new learning module and click 'import questions'. Copy and paste the existing questions from your original assignment's exam into this new learning module. Double check when the new learning module is open to students on the setting page and publish it!
Q: Do you have a template to set this up for me?
A: Sure, just email questhelp@austin.utexas.edu and let us know in what class(s) you'd like it dropped in (and with what name). So you're empowered to make others needed throughout the rest of the semester, please note a few features below when making a new assignment for it to be an online test.
Type: learning module
Start date (when you want students to have a live active link)
Due date (end window of access—note, the window can be much larger than the actual time of the exam due to a time feature*)
Convert numeric questions to multiple choice (optional--if you want enabled. In learning modules this is not necessary—numeric free response is supported)
*optional time limit (minutes)-a timer will start once a student has progressed off the initial instructions slide and once the time has expired the student will not be able to submit any more answer or see the exam.
Test mode-check this. This means students will only have one entry per question and will not receive immediate feedback (ie if they got it right or wrong)
Overview and Class instructions-as you wish
Q: Can I preview assignments as a student before showing it to the actual students?
A: Yes, there are two ways to do this. (1) Without any additional assistance needed, you can set the start and due date to be the exact same day/time, just a minute or two from the present moment. Then publish the assignment, and give yourself an extension. Toggle over to student in the upper right hand corner and you will have student view of the assignment without any other active students in your class having access to it (although they will see the name of the assignment without hyperlinks).
(current)
(forthcoming ~ May 20)
Note: as the primary instructor you are automatically enrolled in your own class as a student. Your TA's and other listed instructors are not–they will need to be added as students as well.
Alternatively, if you have a sandbox course where there are no students, or the 'students' are all colleagues, just publish the assignment with timing so you can access it, since it's not a concern that other eyes will be on it.
(2) Quest does support a staggered start time by unique, where you can be in the front-running section days or hours before your students see it. If you'd like to do this please send us a note (questhelp@austin.utexas.edu) to activate this feature for your class; we'll make sure only you (and any TAs you indicate) have access to the early start time, and you can set when you'd like the assignment to open for you on the setting page.
Q: Can I have an online exam (ie learning module in test mode) that is more than multiple choice? A numeric free response? Short answer? Essay? Grids/matrices?
A: Yes, yes, yes, all around yes! Hybridized tests that incorporate different types of questions are a great way to assess students on a wide range of learning objectives, while also saving yourself time. Right now, for any manually graded elements create the question as an essay question (which are automatically graded manually).
Q: I have Quest automatically drop the lowest exam in my gradebook, but now I need to make another category for online exams. Will the system know to drop the lowest score in these two separate categories (exams, online exams/learning modules in test mode)?
A: Yes, we can roll online exam scores into your previous exam category, so the dropping and averaging machinery will work. Please contact us (questhelp@austin.utexas.edu) for more details.
Q: When giving online exams using Quest, can students log off and come back on?
A: Yes, as long as students are within the window of time allotted for the test, there is no problem with this on the Quest side. If using a Proctoring overlay, that behavior may be suspect and can be communicated ahead of time not to flag.
Q: UT copy used to request 48 hours to get the printed copy back to me. With online exams, does that mean I can make it the day of administration?
A: Please still publish your exam 24-48 hours before administration. This gives us a window to ensure all questions will render correctly and speedily. It will also give us time to troubleshoot should there be any system maintenance interference.
Q: I hear Quest and Canvas can talk to each other. How do I get that set up?
A: Steps to be taken are here; please send a note to questhelp@austin.utexas if any links are not appearing (we need to give you an additional permission). Please note that if the name of the Quest Assignment Type where you've created the assignment (ie in the settings, listed as an exam/homework/learning module type)— matches the name of the Canvas Assignment Group, the assignment should automatically get filed in the correct spot when it's transferred to Canvas–there is no need for you to create the specific assignment in Canvas first.
Q: How can my students see their grades?
A: Right now, it's a bit buried on student view–grades, online tests (or learning module, whichever is most relevant), in the section. However, by taking the assignment out of test mode students can see their grades on their initial page within a class.
Q: What about solutions? Are they posted? Not?
A: The default behavior for test mode is to not release solutions. Right now there is a bug and solutions aren't displaying at all; we're working on it and hope to have a fix out soon.
Student facing
Q: Are units needed on numeric free response?
A: No, if entering in anything but numbers it will be counted incorrect (even if correct)
Q: Can students review and change their submissions after clicking 'submit answer', like they can on a bubblesheet?
A: Yes, up until there is an input for all questions–at that point no more changes are possible (even if there is sufficient time left).
Q: Can I wait and submit all my answers at the end?
A: That approach is really not recommended, although yes, it is possible. The timer cutoff is not well signposted and one could easily lose points from not entering answers in that you are all prepared to. Quest staff is usually available at the start of exams for monitoring and troubleshooting as well, as we have a vested interest in ensuring all goes smoothly
Recommendations
Be overly explicit on what is and what is not permissible for this exam in the directions (ex: "This is an online-only exam. Close all other window and tabs before opening the exam, and do not use any unapproved external resources to complete this assessment. You may use XX. Once you start, you have up to XX minutes to complete the exam.
As you take the exam you and your keystrokes will be monitored virtually; this requires access to your web camera. Confirm your video camera usage before proceeding to slide two--you will be notified when the online proctoring is initiated; a small video box of your live feed will be displayed in the corner and is verification all is working well (Quest variety). If for any reason you are not prompted for webcamera feed please reach out to your professor and questhelp@austin.utexas.edu
You can move through the exam problems in any order you would like, using the > arrows to navigate between the slides. An answer is recorded once you click the blue 'Submit Answer' button and receive a 'Your response was received' note in its place. Please note, if you select an answer choice and navigate to another slide without clicking the blue 'submit answer' button, your selected answer choice will not be saved when you return to this slide. In such a case you can navigate back to the given question and submit a solution. Answers cannot be changed once you have submitted your answer for a question. As such, we recommend only submitting answers on problems you are ready to answer, and leaving the others blank until you are ready to address them.
The results page indicates (with a *) if you have any questions that you have not submitted responses for. Please answer all questions to the best of your ability.
Good luck!")
Make the first question an acknowledgement of UT's honor code (question #219393)
Consider pooling questions, especially for the first few.
*Proctoring Tools
Quest in house (under development): Snapshots of a proctoring video feed will be composed into a flipbook and offered to instructors to uphold academic integrity concerns. Enabling or disabling this feature is expected to be on the specific assignment settings page. More information will be posted on Quest Proctoring capabilities here as it is created.
An UT Arlington professor (with a class of 80) watched students via Zoom: Create a Zoom meeting during testing time, requiring students turn their cameras and mics on. You will then be able to watch them and have full control over muting/unmuting the mics to ensure student's are not interacting. If UT Austin class size concerns are much bigger than this, consider breaking up the class alphabetically into different zoom section invites (and/or having a team of TAs to also monitor).
Approved by UT information security office (iso):
Protorio (the school of nursing, SON, has used this extensively)-a chrome browser add on (ie students must download software to their computer) that offers lock down and remote proctoring features. SON offers informational resources, including a video tutorial here (walk through of assignment in Canvas starts at minute 17, security features in Canvas start around minute 24). The Canvas Training Center also offers this info.
proctorU (vendor contact Georgia Hux)
