Format for shapefile contribution

[ar-puuk]

I spent some time yesterday trying to download some files from archive.org's mirror of the FTP server that includes census block shapefiles. I have them as individual zips. Would you like me to help push the extracted shapefile or zipped folder would be okay?

[ar-puuk]

nvm, some files are bigger than 100MB and it wants me to install Git LFS for it. Unsure if it would be compatible with the main branch.

[vengroff]

Thank you.

I have a branch with LFS and the only reason it's not merged yet is I can't push it because I don't have enough LFS bandwidth.

batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

I bought another data pack but GitHub seems to taking its own sweet time charging my credit card and activating the additional bandwidth.

This branch also has more details on contributing. Watch for it soon.

[vengroff]

OK, it went through.

[ar-puuk]

I will try it later today.

Now that the FTP is up, how would you like to approach it? Would you want this repo to be a mirror of the entire TIGER/Line server, or do you see it being something else?
I am unsure what is the size of the entire data. It might be expensive to host those data here in the long term.

Thanks a lot for all your hard work.

[vengroff]

Probably going to be a bit of wait and see for the next few days. This project was an emergency response when we had no idea how long the outage would last. Basically it's optimized for enabling people to contribute their censusdis caches. It is definitely not cost optimized for the long term.

So I'm going to let it sit as is and explore other mirroring options. Sizewise I'm guessing a complete mirror would be less than 100GB but not a lot less.

[ar-puuk]

Understood. Let's keep monitoring the system, and see if there are any reports of data that were missing/modified since the takedown and we can work towards it.

In the meantime, I am trying to mirror the census (including TIGER/Line) and lehd data from the FTP server, but it is not working either in Filezilla or WinSCP. I do not know if this is a me issue or the server issue. Have you tried doing so recently?