-
Notifications
You must be signed in to change notification settings - Fork 25
Description
During an exam, students cannot see the results of penalty code checks or penalty meta tests. It would be nice if we could inform the student that they are getting such a penalty (when they click "Only tests").
For example, the description might say "do not use filter", but if the student misses this, they'll lose all points, without Andy telling them that this will happen.
We still need to determine whether we should make all penalties visible, or only a subset.
- For concrete penalties that are mentioned in the description, like "do not use
filter" (in a PBT assignment), it always makes sense to show them. - For penalties like "
getTripByIdshould not be verified" (in a mocking assignment), we probably do not want to show them. (They check whether students meet the learning objective of "being able to determine whether something should be a stub or a mock".) - For penalties like "Meta test: finds bug where very specific indices are missing (fails when tests do not cover the input space well) (penalty: 50)" (in a PBT assignment), we still need to decide. (They check whether students follow the instruction to write "good" property-based tests, but they're not completely fool-proof, so they will cause problems because of false positives and false negatives.)
We probably want a mechanism where we can decide for each individual penalty whether it should be visible when clicking "Only tests".
We should be careful not to introduce vulnerabilities. If I remember correctly, the RunConfiguration is destroyed for "Only tests" so that students cannot access it. If we need to change this, we should be careful not to open up too much.