Loss Function weights to replicate the results as shown in the paper

[JS2498]

I tried to replicate the results by running the code with model specifications given in the paper. The value hyperparameter lamdba that is used for combining the chamfer and regularization loss is not given. In the code it is mentioned that the weights for these losses as chamfer_weight=0.85, regularization_weight=0.15, are these the weights used in obtaining the results presented in the paper?

Can you please also post the plots of loss v/s epochs ?(will be helpful for cross verification)

[Arritmic]

Hi,

Thanks for testing the code and bringing this issues to my attention.

Right now, I cannot access to the servers to check exactly the logs of each model (additionally, we have trained quite a lot with different variations in the arch), but next week I will try to provide you more specific information about the model we used for the results. Additionally, one of my colleagues, it is right now training new versions (with new loss functions), and he just told me 2 days ago that he was able to reproduce the results. But if I do not remember bad, we were using Lambda=0.1 or even less. And for sure, you need a lot of iterations until starting to see the generalization.

But, what is the problem exactly? What results are you not able to replicate, qualitative or quantitative? I am testing right now at my home the code, training the model with a little subset of the database, and it looks like this, as it use to be in the training in the supercomputer:

By the way, have you tested the inference with the provided model? We are training new models, and we will share soon also new models and mods of the shared arch.

[Arritmic]

By the way, the model we have shared and tested has this configuration:

    model = CSI2PointCloudModel(
        embedding_dim=512,
        num_heads=4,
        num_encoder_layers=4,
        num_decoder_layers=4,
        num_points=1200,  
        num_antennas=3,
        num_subcarriers=114,
        num_time_slices=10
    ).to(device)

[JS2498]

But, what is the problem exactly?

• The predicted point cloud for different activities is almost same (qualitative) when I trained and tested the model. I am not able to replicate the visuals (plots of ground truth and predicted PCD as in the video). Hence trying to understand where I might be wrong.
• The below results is obtained using the model configuration mentioned in the paper trained for 30 epochs

[Arritmic]

Hi Jayanth,

Yes, something is happening there. And yes, I was sure that you were referring to the qualitative results and issues related to the dynamics and temporal motion. Two questions:

• Have you tried the testing with the provided model? Does it work for you?
• If you remember, the checkpoint that I have uploaded is from the Epoch 41, and these models started to show generalization in terms of "motion" detection in higher epochs. But I am having right now some doubts about the configuration we shared in the Github. Even the model is exactly the same, I am having doubts with two things: the scheduler, and the loss function. If you took a look into the loss, there are two different implementations of the Chamber loss. Right now, one is commented. So, I am not sure if we used one or another (even I believe that I was carefully checking that, it could be that with the rush I would put the wrong Chamber Loss). So, please, give me a while and I will try to give you as much precise info as possible about these issues, and share the train and validation loss errors.

[Arritmic]

Hi @JS2498!

I have found the issue. Everything was correct, including the hyperparameters, but finally, in the loss function, in the feature regularization, we have made a normalization of the transformation matrix (the features where too big for the loss part from the chamber loss), and I forgot to add that when moving the code from the server to the public code. But I will do it soon. As you can see, the results are as follow with that normalization:

Find here, the sequence of several frames in the epoch 20 from one subject not included in the training:

[Arritmic]

New Loss function available at: https://github.com/apm-loss/apml