About the metric MSE

[llan-ml]

Hi @yunshengb , thanks for releasing this repo.

I found that you use a denominator of 2 in the function of mean_squared_error, which I don't think should be used when reporting the metric of MSE.

It would be better to declare this in the readme file or update the code.

[yunshengb]

Thanks for your note. Indeed, we use a denominator 2 in the evaluation. We encourage you to reuse our code for reproducing the results, and please let us know any other confusion or questions you have. Thanks!

…

On Dec 13, 2020, at 02:20, lanlin ***@***.***> wrote: —

[llan-ml]

@yunshengb
Thanks for your reply.

I have another question about the training convergence. I found that the convergence results across different runs with the same configuration vary a lot as shown in the following figure:

I guess that the reason could be different runs are with different random seeds, which further influences the sampling of training data and the initialization of weights. However, the variance across different runs is quite big.

Do you also encounter this problem and do you have any advice to alleviate this?

[rishabh-ranjan]

Thanks for your note. Indeed, we use a denominator 2 in the evaluation. We encourage you to reuse our code for reproducing the results, and please let us know any other confusion or questions you have. Thanks!
…

Why is the denominator of 2 used in the evaluation? This has not been documented in the paper. My concern is that the numbers in the paper under the heading of MSE are misleading, since the standard definition of MSE does not have this factor of 1/2. Please clarify.