Before we talk about the best way to swap questions within an exam, it is important to understand an exam mechanic in OASIS called a dynamic exam. So what is a dynamic exam? To demonstrate, the easiest way to start is to start with what is a normal exam, which actually is a static exam. The reason we call it a static exam is because the exam questions are started with 10, and every student will get the same 10 questions. In this illustration, we happen to have a fixed sequence, but obviously each student's exam can be randomized. But all you're doing is you're randomizing the sequence of the same 10 questions. So everyone gets the same 10 questions. It's just a matter of whether you want to scramble the order of the questions or not. But this is, I would say, 90% of the cases where you create an exam with 10 questions, and everyone gets the same 10 questions. Now, there is a mechanic called dynamic exam, which means you set a question pool of 15 questions, and then you say everyone gets 10 out of the 15 in the pool. So in this illustration, you'll see that everyone gets 10 questions, but which 10 questions you get is entirely random as long as you get 10 questions out of the 15. So this is the mechanic that is really designed so that when you have more questions than is necessary for the exam, or you want to increase the security of the exam, you can overload your question pool and have a lesser required questions in the student's exam to create a further randomization so that each student's question is not only different by the order of the question, but the actual questions is also different. Obviously, to achieve this mechanic, you will have to have more questions than is needed. So you cannot do this if you only have 10 questions and every student needs to have 10 questions on the exam. Now, when you have a dynamic exam, what you can do then is you can modify the pool over time. So in this illustration, as you can see, I have a 15-question pool, except that in the beginning, which is from the previous slide, I have question 1 through 15, and let's say one day I decided to remove five questions out of the pool, and I'm going to add five more questions in the pool. So I still have 15 questions in the pool, but which 15 has changed? Obviously, it doesn't have to stay at 15. You can remove five old questions and add 10 more questions. That doesn't matter, right? But in the screenshot here, I'm only illustrating the fact that I'm removing five and adding five. When you do that, it is the exact same exam. It's just that there are two cohorts, essentially. You have one cohort, we call it A, that took the exam before the modification. So you can see the questions are frozen in time where the questions are randomized, randomly pooled from question 1 through 15. And then you also have cohort B, which are students who have started the exam after the modification. And notice that the questions for cohort B are only from the pool 6 through 20, right? So question 1 through 5 is no longer in the questions that cohort B has because they started the exam after question 1 through 5 has been removed from the pool. I want to emphasize here that the question that's going to be used in a particular student's exam is frozen based on when they started the exam. So there is no such thing as what happened to the student who haven't finished the exam when I made the change because it is frozen at the time of the start. So even though they haven't submitted the exam, if the student cohort A started the exam but hasn't finished, the questions will still stay with the question that was in the pool when they started the exam. So you could have some student who haven't submitted the exam yet, but by the time they submit the exam, the questions in their exam is no longer as a part of the pool, which is okay. OASIS does not care about that. OASIS is only taking a snapshot of what is in the pool at the time of when the user start the exam. This allows you to constantly replenish your question pool with more questions and remove questions that is no longer relevant or no longer fresh. And you can also increase the question pool from 15 to 20 to 25 to 100. And knowing that as you add more questions to the pool, the people that's benefiting from a bigger pool are the people who is going to start the exam after the addition of the new questions. Now, this mechanics under dynamic exam is critical for how we can do a massive amount of question swaps in a particular assessment exam. Before we end this concept, just a fringe case scenario is if you have a 10-question pool and the student is supposed to get 10 questions, then a dynamic exam is the same as a static exam because you're telling the system to pull 10 questions from a 10-question pool, which essentially is a static exam. In the next video, we'll talk about how to leverage this dynamic exam behavior to do a bigger change, bigger swap of questions after the exam is in flight.

dynamic exams

OASIS platform

question pool

random selection

static exam