[DataCap Application] <Han Tang Cloud> - <Research on the impact of climate change on agriculture based on cloud technology>

[ChenggB1n]

Data Owner Name

XSG

Data Owner Country/Region

China

Data Owner Industry

Environment

Website

http://www.htymeta.com/

Social Media Handle

1411479497@qq.com

Social Media Type

Other

What is your role related to the dataset

Data Preparer

Total amount of DataCap being requested

3PiB

Expected size of single dataset (one copy)

500TiB

Number of replicas to store

6

Weekly allocation of DataCap requested

1000TiB

On-chain address for first allocation

f1qwzlpo6xzf67zpuhyohtilagejy4rlqobxgk33y

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

• Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

Project: The main research is on the impact of climate change on agricultural production, and the impact of temperature, precipitation and extreme climate events on crop yields. We will use climate data from public datasets and Han Tang Cloud's cloud computing and big data processing capabilities to establish accurate prediction models, provide scientific basis for agricultural management and policy making, and strive to provide technical support for the national East-West computing strategy. Understanding the specific impact of climate change on agriculture can help formulate effective response strategies. We hope to promote the development of sustainable agriculture through this project, combined with distributed storage and big data technology.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

Download public data using NASA's Atmospheric Science Data Center, an archive of NASA satellite and model data related to atmospheric chemistry, atmospheric dynamics, precipitation, hydrology, solar irradiance, weather, and climate.

Where was the data currently stored in this dataset sourced from

My Own Storage Infra

If you answered "Other" in the previous question, enter the details here

No response

If you are a data preparer. What is your location (Country/Region)

China

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

We have our own storage facility. The data is based on NASA open source big data warehouse. After downloading, use filedrive-team's open source go-graphsplit to cut it into piece files of specified size. The csv file will store the piece_cid and size information corresponding to the source file. We will also use but not limited to Droplet and other tools.

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

No response

Please share a sample of the data

Climate research data in public datasets, especially MODIS datasets, the following are some samples：
https://search.earthdata.nasa.gov/projects?p=C2408009906-LPCLOUD!C2408009906-LPCLOUD&tl=1729159597!3!!&fst0=Atmosphere&lat=33.73861309212248&long=43.119140625&zoom=5
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL1BRAD.001/EMIT_L1B_RAD_001_20241016T075314_2429005_022/EMIT_L1B_RAD_001_20241016T075314_2429005_022.nc
s3://lp-prod-protected/EMITL1BRAD.001/EMIT_L1B_RAD_001_20241016T075338_2429005_024/EMIT_L1B_RAD_001_20241016T075338_2429005_024.nc

Confirm that this is a public dataset that can be retrieved by anyone on the Network

• I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Daily

For how long do you plan to keep this dataset stored on Filecoin

More than 3 years

In which geographies do you plan on making storage deals

Asia other than Greater China, North America

How will you be distributing your data to storage providers

Cloud storage (i.e. S3), IPFS, Shipping hard drives

How did you find your storage providers

Partners

If you answered "Others" in the previous question, what is the tool or platform you used

No response

Please list the provider IDs and location of the storage providers you will be working with.

f03179570 SP     
f03178077 JP    
f03179572 US
f03214937 US

f03298672 DE
f03298673 DE
f03308933 UK
f03308920 JP
f03308922 JP

f03385462 US
f03385456 US
f03402285 CN
f03402308 CN
f03402347 SP
f03402379 SP

How do you plan to make deals to your storage providers

Boost client, Lotus client, Droplet client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

[datacap-bot]

Application is waiting for allocator review

[martplo]

Hi, @ChenggB1n Thank you for your application. Please explain how the data is transformed into deals for filecoin. Could you include some documentation?

[clriesco]

?

[martplo]

@clriesco sorry :)

[ChenggB1n]

Hi, @ChenggB1n Thank you for your application. Please explain how the data is transformed into deals for filecoin. Could you include some documentation?

First, we will ensure that the data meets the storage requirements of the Filecoin network. We will package the source data that needs to be encapsulated into a tar file and use graphsplit to slice it into files of the specified size and in CAR format, ensuring data integrity. The information of the sliced CAR files will be stored in a CSV file. For each packaged CAR file generated, we will record the dataset name where the source data file is located, as well as the piece_cid, payload_size, and other relevant information for that CAR file in the CSV file. This will ensure the complete preservation of the dataset and document which storage providers (SP) packaged which parts of the dataset. We will submit the order information to the database using either the droplet-client or an offline order submission method. Subsequently, we will use Filecoin network tools (such as Lotus and Venus) to create storage contracts that convert the data into Filecoin transactions.

[willscott]

how will these metadata CSV files be made available?

[ChenggB1n]

how will these metadata CSV files be made available?

When using a program to slice files, a manifest.csv will be automatically generated, and a script will automatically record the already sliced source files to avoid data duplication, as shown in the following example:

[willscott]

how do i download all of the manifest.csv files so that i can understand what is in which deals?

if you don't put those on a public website, i can't reconstruct the dataset as a client.

[ChenggB1n]

how do i download all of the manifest.csv files so that i can understand what is in which deals?

if you don't put those on a public website, i can't reconstruct the dataset as a client.

Are you saying that if I want to use this DC allocation, I need to submit the complete CSV file each time after sealing?

[willscott]

i'm saying they need to be put on a website or ipfs or otherwise made available in some way so that a public client can have a chance of reassembling this supposedly open data.

[ChenggB1n]

i'm saying they need to be put on a website or ipfs or otherwise made available in some way so that a public client can have a chance of reassembling this supposedly open data.

I see what you mean. I will put this file on a public network drive to ensure that other users can download it and have the opportunity to reassemble this open data.