hints based on check.call

[MartinKies]

Could you give me a pointer regarding the usage of hints when working with check.call instead of check.assign and check.function?

Say I want the student to use a two-sample t-test (but expressively not a Welch-test!). The following workaround does the trick:

#< task
#Write your solution here.

#>
my.test <- t.test(column1, column1, var.equal=TRUE)
#< hint
if(true(identical(my.test,t.test(column1, column2)))){
  cat("You have used the Welch-Test. Use the option var.equal=TRUE" )}
#>
my.test

The automatic hint directs the student to the missing option, but I think it is helpful to give context information.

It would be nice however if I could write something like

#< task
#Write your solution here.

#>
t.test(column1, column1, var.equal=TRUE)
#< hint
if(true(identical(<expr_student>,t.test(column1, column2)))){
  cat("You have used the Welch-Test. Use the option var.equal=TRUE" )}
#>

This way the student would not have to do the artifical step to first save and then display the variable, as we are only interested in interpreting the output.

Is this possible/did I miss this in the manual?

[skranz]

That is possible in principle, but I am not sure whether it is a good idea to develop such sophisticated custom hints. Over time I tended to make simpler exercises using #< fill_in blocks. So for example, here I might write:

#< fill_in
#Write your solution here.
t.test( ___ , ____, var.equal=___)
#>
t.test(column1, column1, var.equal=TRUE)

And then don't add a customized hint. Or I would add in the hint just a general note that if var.equal = FALSE, we have the wrong test, not checking the students solution.

But of course if you really want to polish your problem sets such customized hints can be done. Here is an example:

#< task_notest
# Create a sequence from 1 to 10 with step size 2
#>
seq(1,10,by=2)
#< hint
restore.point("in_my_hint")
val = try(eval(parse(text = .stud.code)), silent = TRUE)
if (true(identical(val,seq(1,10,by=1)))) {
  cat("You need to add the argument 'by=2'")
}
#>

The code in the hint will be evaluated in an environment that contains all the variables created in previous chunks and that have been generated when the students code in that chunk has been run. In addition there is a special variable .stud.code that contains the student's code as string. Here I just evaluate that code and then compare the returned value with the value from the alternative call. That would of course be more complicated in a chunk where a student shall enter multiple commands.

Note that you can use restore points to debug your custom hint.

[MartinKies]

Thank you very much for your fast answer! Yes, this works.
There is of course always a time/effort consideration to be done, but as a longterm vision I think it would be really cool if the student would have the feeling that he actually has a Tutor beside him, who gives him detailed feedback. It is of course never possible to catch all possible inputs, but I think if one has a problem set which e.g. gets frequented by dozens of people each month it should be possible based on the tracking of the inputs to quickly develop customized hints for the most common errors a la Pareto Principle :).

[skranz]

Sounds actually like a good idea that I can also use for my courses. I am working on a more convenient function to show such conditional hints and an integration with RTutorSAGI to generate some automatic templates for such conditional hints based on student's common mistakes.

[MartinKies]

Oh yes, I think that would be an amazing addition to the package!

[skranz]

Ok, the RTutor part is now implemented with this commit that introduces the function hint.stud.call and some more.

Take a look here for documentation and examples.

If at some point you test it out, please let me know whether it works nicely.

I will now (or later) check in how far one can create some nice hint skeleton code with RTutorSAGI.

[MartinKies]

Thank you very much!
All current use cases I have tested so far work nicely.

Even this works:

#< task_notest
#Führen Sie hier die Aufgabe aus. 

#>
#< task_notest
# Draw a vector of 10 normally distributed random variables with mean 0 and variance 4. Afterwards draw a second vector of 10 uniform random variables with min 0 and max 5.
#>
rnorm(10,0,2)
#< hint
hint.stud.call(rnorm(10,0,4),"Remember that R expects the standard deviation, not the variance, as 3rd argument of rnorm.")
auto.hint.else()
#>
runif(10,0,5)
#< hint
hint.stud.call(rnorm(10,0,5),"We wanted a uniform distribution!")
auto.hint.else()
#>

which is not obvious based on the vignette and might be useful to know to new users.

Is there a specific reason why you opted for having to call auto.hint.else()? The auto hints are generally extremely useful and so I am pretty hard pressed to think of reasons why one might want to have them off. I expect that I will put the auto.hint.else() after each hint.

Of course I could otherwise use add_to_hint but this shows a different behavior if the expected erronous call is executed as now both the auto hint and my text are displayed.

Design-wise I would suggest to leave auto-hint always on, and have a function suppress.auto.hint() (or something like that) if one explicitly only wants the hints which one has generated oneself.

[skranz]

I just made another update that also adds the function hint.else for a customized alternative hint. The documentation is also updated, with a complex example here

Taking this and other aspects into account (like flexible positioning), I like the explicit auto.hint auto.hint.else or hint.else calls more.

[skranz]

Ok, all features I thought about so far, are now included in RTutor and RTutorSAGI:

• RTutor now also has a function hint.stud.fun that allows an adaptive hint if the student called a specific function, independent of the function arguments. There is an example in the Hint & Test vignette.

• RTutorSAGI now has functions that can create from a set of submission files a data frame that contains code templates for adaptive hints based on common errors in the submitted problem sets. Example code to use it is here. Documentation is sparse, but I hope it is self explanatory if you try it out (of course you need submission files).

[MartinKies]

Thank you very much! I have incorporated them in my fork an will test them. If I find anything, I come back to you!

For the RTutorSAGI Tests I will probably need a month however, as this will be relevant for us, when we get the results of this month.