cannot copy data issue

[rossdwilliams]

getting a strange error running the execute function, seems to generate cohorts and load them but then i get the following:

2021-09-16 11:39:20 [Main thread] INFO FeatureExtraction Fetching data took 30.6 mins
2021-09-16 12:04:41 [Main thread] FATAL RSQLite connection_copy_database Error: Failed to copy all data: not an error

I thought it might be a space issue because when i sampled 100k instead of the full 1.7m patients this error didnt appear. I am working in a drive with ~850gb but the c drive has only 25gb free. The package and R installations are on the bigger drive as is the options(andromedaTempFolder).

It seems to occur when it tries to copy all the data

This is the output of cohortCounts
"cohortDefinitionId","cohortName","cohortCount","personCount"
2916,targetCohort,1648826,1388549
2918,outcomecohort,33524,28465

[lhjohn]

I have the same issue for cohorts larger than 1 million, but only tested this on PDW database.

[msuchard]

some colleagues have reported similar issues to me in CohortMethod (when grabbing FeatureExtraction data object to build PS models), so I suspect the issue lies in pulling down large data objects.