Batch correction options when running across multiple samples

[Nick-Eagles]

Hello,

Following up a bit on my work related to #30, I was able to run FICTURE, sharing factor assignments across multiple Visium HD samples. I noticed that some factors were sample-specific, which perhaps is expected, but made me concerned about batch effects. From this comment, I saw you have used FICTURE similarly across multiple samples. Did you happen to perform any sort of batch correction, and if so, is there code you're able to share? Otherwise, are there strategies you'd recommend for addressing batch effects in this scenario?

As always, thanks for developing this great software and for your help on previous issues.

Best,
-Nick

[Yichen-Si]

Hi Nick,

I'm sorry that I don't have much experience in batch correction and it is indeed a limitation we hope to work on in the future. I don't have any good idea now... Since there are already many single cell batch effect correction methods, maybe you can create a grid of the Visium HD data and run any of the exiting methods. After that, if you still want to use FICTURE to create a finer resolution visualization you could try the following: after batch effect correction and clustering, create a pseudobulk of each cluster in each sample. Use the sample-specific pseudobulk as the "model" matrix in ficture slda_decode.

Thanks for sharing your comments!

[Nick-Eagles]

Thanks for this suggestion! We're particularly interested in the fully unsupervised implementation of FICTURE, which has performed quite well compared to other clustering options, so we'd prefer not to cluster in advance. Suppose we want to achieve batch correction by perturbing the inputs to the fully unsupervised FICTURE pipeline. Is this statistically reasonable / will any assumptions by the algorithm be violated here? In particular, we were considering a couple distinct options:

1. Library-size-normalized input counts (so just scaling down by total counts in each bin)
2. Batch-corrected counts (that generally preserve the distribution of each gene's counts)

For example, are integer input counts required? Thanks again for your help.

Best,
-Nick

[Yichen-Si]

Great points!
Integer input is not required! (We are working on related heuristics and will let you know when we have more convenient support for that.)

[Nick-Eagles]

Thank you! Unless this was fixed recently (I haven't pulled the last couple commits), it looks look integer counts are actually causing issues. Here's the full error message when ficture make_sge is called from within ficture run_together. Note column 3 (the 4th column) here is Count, and I passed floats.

Traceback (most recent call last):
  File "parsers.pyx", line 1161, in pandas._libs.parsers.TextReader._convert_tokens
TypeError: Cannot cast array data from dtype('float64') to dtype('int64') according to the rule 'safe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/jhpce/shared/libd/core/ficture/dev_a455e5c/ficture_env/bin/ficture", line 8, in <module>
    sys.exit(main())
  File "/jhpce/shared/libd/core/ficture/dev_a455e5c/ficture/ficture/cli.py", line 44, in main
    function(sys.argv[2:])
  File "/jhpce/shared/libd/core/ficture/dev_a455e5c/ficture/ficture/scripts/make_dge_univ.py", line 99, in make_dge
    for chunk in pd.read_csv(args.input, sep='\t', chunksize=1000000, dtype=dty):
  File "/jhpce/shared/libd/core/ficture/dev_a455e5c/ficture_env/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1843, in __next__
    return self.get_chunk()
  File "/jhpce/shared/libd/core/ficture/dev_a455e5c/ficture_env/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1985, in get_chunk
    return self.read(nrows=size)
  File "/jhpce/shared/libd/core/ficture/dev_a455e5c/ficture_env/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1923, in read
    ) = self._engine.read(  # type: ignore[attr-defined]
  File "/jhpce/shared/libd/core/ficture/dev_a455e5c/ficture_env/lib/python3.9/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 234, in read
    chunks = self._reader.read_low_memory(nrows)
  File "parsers.pyx", line 850, in pandas._libs.parsers.TextReader.read_low_memory
  File "parsers.pyx", line 921, in pandas._libs.parsers.TextReader._read_rows
  File "parsers.pyx", line 1066, in pandas._libs.parsers.TextReader._convert_column_data
  File "parsers.pyx", line 1169, in pandas._libs.parsers.TextReader._convert_tokens
ValueError: cannot safely convert passed user dtype of int64 for float64 dtyped data in column 3

I understand this is non-standard use of FICTURE, so I don't mean to bother you further, but perhaps this is still helpful to know.

Best,
-Nick

[Yichen-Si]

Hi @Nick-Eagles ,

I see, I made a small change, if you install the latest version (either with pip or pull from the main branch and build locally), you can add a flag --fractional-count 1 to run_together (or make_sge and fit_model if you use them separately) to resolve the issue.
(Note that we have not tested applying gene level weights in real data, but I'm curious about what you will find out).

Thanks,
Yichen

[Nick-Eagles]

Awesome, I'll use that option. Note that it now seems like running without any --fractional-count option creates an error in make_dge: make_dge: error: argument --fractional-count: invalid int value: 'None'. Specifically, I was running ficture run_together --all. Perhaps the default should be --fractional-count 0?

[Yichen-Si]

Ah thanks so much!!

[camelest]

@Nick-Eagles Hi, I'm also interested in integrating multiple samples in FICTURE. Would you mind sharing some updates on your trial mentioned above? Thank you so much in advance.

[Nick-Eagles]

Hi @camelest, we opted for reading our Visium HD data into R as a SpatialExperiment, performing batch correction there, then exporting counts in the format required by FICTURE. Spatial coordinates were staggered and placed such that each sample is adjacent (with a small buffer), so that FICTURE would treat the many samples as one and find shared factors. Unfortunately the code is private for now, but hopefully this conceptual overview is helpful.

[camelest]

@Nick-Eagles Thank you for your prompt responses. I really appreciate you taking the time to share the overview — this is already very helpful! I will try to combine the datasets using count data after batch correction.