# Upload from API/S3

This technical guide is designed to provide users with instructions on uploading data to VaultNode. It encompasses technical documentation and step-by-step procedures for transferring data from AWS S3 to the Filecoin network through VaultNode. Feel free to utilize this guide to initiate the process of uploading your files to our storage system immediately!

Link: [https://s3api.vaultnode.co](https://s3api.vaultnode.co)

### 1. VaultNode Technical Overview <a href="#houaa" id="houaa"></a>

![](https://3682208887-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FX7whvW6ykW9dj7FtJFwJ%2Fuploads%2FL1V6PHOc5kz1LQdt9uuX%2F0.png?alt=media)

1. **Data Backup and Recovery:** Covers the needs of cloud service providers and enterprises, providing data backup, recovery, and disaster recovery solutions.
2. **Regulatory and AI Data Management:** Ensures regulatory compliance in data storage, and offers efficient data backup for AI training.
3. **Big Data and Autonomous Driving:** Facilitates data security for big data scenarios and autonomous driving data storage.
4. **Healthcare and Security Data:** Offers reliable storage for medical imaging and surveillance data.
5. **User Data and Research:** Handles user behavior/transaction data and scientific research data.
6. **Document Management and IoT:** Supports digital archive management and IoT device data applications.
7. **The VaultNode Advantage:** Offers secure, reliable, and cost-effective data storage across all these scenarios to drive business innovation and digital transformation.

#### 1.1. Overall Usage Process <a href="#yyfdt" id="yyfdt"></a>

![](https://3682208887-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FX7whvW6ykW9dj7FtJFwJ%2Fuploads%2F7NIqIt8pxHfS7XKm9sTG%2F1.png?alt=media)

#### 1.2. Filecoin DataCap Sealing process <a href="#uc5e53e24" id="uc5e53e24"></a>

1. **File Encryption:** The client first encrypts the file to ensure data security. This step is carried out by the client themselves to protect their data from unauthorized access during the upload and storage process.
2. **File Upload:** The client uploads the encrypted file to Vaultnode object storage via an S3-compatible API. This step can be carried out using various S3-compatible tools, such as rclone, etc.
3. **File Management:** After uploading the file, the client can manage their files using S3-compatible management tools (like rclone). These tools can help clients easily view, manage, and operate their files in Vaultnode object storage.
4. **File Encapsulation:** After receiving the file, the platform dispatches an encapsulation task to encapsulate the data to a designated node. This step is automatically completed by the platform, and the client does not need to perform any operations.
5. **Data On-Chain:** After the encapsulation is completed, the data will be put on the Filecoin network. Clients can view the corresponding on-chain information of the data through the node, including the location of the data, status, etc.

### 2. Storage AWS S3 Integration <a href="#uadff2897" id="uadff2897"></a>

Describes Vaultnode API operations in detail. Also provides sample requests, responses, and errors for the S3 client tools.

Vaultnode supports all clients that support S3 (AWS CLI、 rclone、 Cyberduck、S3cmd、CloudBerry Explorer、DragonDisk、s3fs)

#### 2.1. AWS CLI <a href="#id-7820fdb5" id="id-7820fdb5"></a>

Vaultnode supports all the clients and tools of the standard S3 protocol, with the AWS CLI as an example of configuration and usage.

How to Use the AWS CLI to Interact with Vaultnode S3 Compatible API.

For more information on AWS CLI, see [AWS CLI Help](https://docs.aws.amazon.com/cli/latest/reference/s3/)

#### 2.2. Prerequisites <a href="#prerequisites" id="prerequisites"></a>

* [Download and install](https://aws.amazon.com/cli/) AWS CLI tools.
* Get Access Key and Access Secret Keys.(Provided separately)
* Use Vaultnode endpoint url: [https://s3api.vaultnode.co](https://s3api.vaultnode.co/)

#### 2.3. Authorized for use by the Vaultnode <a href="#id-75c50ccd" id="id-75c50ccd"></a>

You can contact our engineers by submitting your email application: Authorized for use by the Vaultnode

**Access Key ID** :\<Access Key ID>

**Secret Access Key**:\<Secret Access Key>

**Vaultnode endpoint url**：[https://s3api.vaultnode.co](https://s3api.vaultnode.co/)

#### 2.4. Configure the AWS CLI <a href="#id-364f0777" id="id-364f0777"></a>

aws configure\


The command will generate a series of prompts, fill in the following:

* AWS Access Key ID \[W76I]:  \<Access Key ID>
* AWS Secret Access Key \[G8Ms]: \<Secret Access Key>
* Default region name \[us-east-1]:
* Default output format \[None]:

#### 2.5. Crete a bucket <a href="#id-4f47f51a" id="id-4f47f51a"></a>

To create a new bucket on Vaultnode using the AWS CLI, use the following command:

aws --endpoint-url https://s3api.vaultnode.co s3 mb s3://\<bucket-name>\


For example, to create a new bucket named "mybucket":

aws --endpoint-url https://s3api.vaultnode.co s3 mb s3://mybucket\


Bucket names must be unique across all Vaultnode users, be between 3 and 63 characters long, and can only contain lowercase characters, numbers, and dashes.The terminal should return the following line:

make\_bucket: mybucket\


#### 2.6. Display buckets <a href="#cbf61346" id="cbf61346"></a>

The following command will list all buckets in your Vaultnode account:

aws --endpoint-url https://s3api.vaultnode.co s3 ls\


The terminal should return all currently authorized buckets:

2023-05-02 15:36:38 vault\


#### 2.7. Upload file <a href="#ffeed24e" id="ffeed24e"></a>

To upload a single file, use the following command:

aws --endpoint https://s3api.vaultnode.co s3 cp \[filename] s3://\[bucket-name]\


For example, to upload a file named "today.txt" to the bucket "mybucket":

aws --endpoint https://s3api.vaultnode.co s3 cp \~/docs/today.txt s3://mybucket\


To verify that this file was uploaded by listing the contents of the bucket with the command used earlier:

aws --endpoint https://s3api.vaultnode.co s3 ls s3://mybucket\


To upload multiple objects, use the following command:

aws --endpoint https://s3api.vaultnode.co s3 sync \[folder name] s3://\[bucket-name]\


For example, to upload the contents of a folder named "docs", use the following command:

aws --endpoint https://s3api.vaultnode.co s3 sync \~/docs/ s3://mybucket/docs/\


#### 2.8. List files in a bucket <a href="#u01c5cc4b" id="u01c5cc4b"></a>

To list the files of a bucket, use the following command:

aws --endpoint https://s3api.vaultnode.co s3 ls s3://\[bucket-name]\


For example, to list the files of 'mybucket':

aws --endpoint https://s3api.vaultnode.co s3 ls s3://mybucket\


#### 2.9. Move a file <a href="#u9b7c4e47" id="u9b7c4e47"></a>

To move from bucket 1 to bucket 2, use the following command:

aws --endpoint-url https://s3api.vaultnode.co s3 mv s3://\[bucket-name1]/\[file-name] s3://\[bucket-name2]/\


#### 2.10. Copy a file <a href="#u650bb9b6" id="u650bb9b6"></a>

To copy from bucket 1 to bucket 2, use the following command:

aws --endpoint-url https://s3api.vaultnode.co s3 cp s3://\[bucket-name1]/\[file-name] s3://\[bucket-name2]/\


#### 2.11. Download a file <a href="#uf6b38f71" id="uf6b38f71"></a>

To download a single file, use the following command:

aws --endpoint https://s3api.vaultnode.co s3 cp s3://\[bucket-name]/\[file-name] /path/to/download/filename\


For example, to download a file named "today.txt" from the bucket "mybucket":

aws --endpoint https://s3api.vaultnode.co s3 cp s3://mybucket/docs/today.txt \~/docs/today.txt\


#### 2.12. Download a folder <a href="#uecde4528" id="uecde4528"></a>

To download a folder, use the following command:

aws --endpoint https://s3api.vaultnode.co s3 cp --recursive s3://\[bucket-name]/\[folder name] /path/to/download/folder\


For example, to upload the contents of a folder named "photos", use the following command:

aws --endpoint https://s3api.vaultnode.co s3 cp --recursive s3://mybucket/docs \~/docs\


#### 2.13. Delete a file <a href="#uaa07863f" id="uaa07863f"></a>

To delete files, use the following command:

aws --endpoint https://s3api.vaultnode.co s3 rm s3://\[bucket\_name]/\[file\_name]\


#### 2.14. Delete all files in a bucket <a href="#udadd5818" id="udadd5818"></a>

To delete all files in the bucket, use the following command:

aws --endpoint https://s3api.vaultnode.co s3 rm --recursive s3://\[bucket\_name]/\


For example, to delete all files from the bucket "mybucket":

aws --endpoint https://s3api.vaultnode.co s3 rm --recursive s3://mybucket/\


#### 2.15. Generate a presigned S3 URL using the AWS CLI <a href="#u1778c858" id="u1778c858"></a>

To create a presigned URL using the AWS CLI, use the following command syntax:

aws s3 --endpoint https://s3api.vaultnode.co presign s3://mybucket/file.name\


This command should return a presigned URL. By default, the expiration time is one hour.You can specify different expiration times by adding flags and minutes. --expires-in

#### 2.16. Use S3-Compatible API <a href="#u8af44735" id="u8af44735"></a>

The Vaultnode S3 Gateway supports a API that is compatible with the basic data access model of the [Amazon S3 API](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html).

**2.16.1. API Endpoint**

The Vaultnode S3-Compatible API endpoint URL is:[https://s3api.vaultnode.co](https://s3api.vaultnode.co/)

**2.16.2. Authentication**

Vaultnode S3-Compatible API supports AWS v4 signature (AWS4-HMAC-SHA256) for authentication, supports AWS v2 signature, v4 signature is recommended.

**2.16.3. Https Protocol**

Vaultnode maintains strict https-only standards. This means that objects and API calls are only served over https. The port for this connection is the https standard port 443 . There is currently no way to disable this feature. Requests sent over the http protocol will be redirected to https.

**2.16.4. Rate Limit**

Vaultnode S3-Compatible API has an effective rate limit of 100 RPS (requests per second).

**2.16.5. Supported API Methods**

| **Method Name**         | **Method Description**                                                                                                                                     |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| CreateBucket            | Creates a new bucket.                                                                                                                                      |
| ListBuckets             | Returns a list of all buckets owned by the authenticated sender of the request.                                                                            |
| DeleteBucket            | Deletes the bucket. All objects in the bucket must be deleted before the bucket itself can be deleted.                                                     |
| ListObjects             | Returns some or all (up to 1,000) of the objects in a bucket. We recommend that you use the newer version, ListObjectsV2                                   |
| ListObjectsV2           | Returns some or all (up to 1,000) of the objects in a bucket.                                                                                              |
| GetObject               | Retrieves objects from DSS. To use GET, you must have READ access to the object.                                                                           |
| HeadObject              | The HEAD action retrieves metadata from an object without returning the object itself.                                                                     |
| PutObject               | Adds an object to a bucket. You must have WRITE permissions on a bucket to add an object to it.                                                            |
| CopyObject              | Creates a copy of an object that is already stored in Oort DSS.                                                                                            |
| DeleteObject            | Delete an object from a bucket. If there isn't a null object, Oort DSS does not remove any objects but will still respond that the command was successful. |
| DeleteObjects           | Delete an object from a bucket. If there isn't a null object, Oort DSS does not remove any objects but will still respond that the command was successful. |
| AbortMultipartUpload    | This action aborts a multipart upload. After a multipart upload is aborted, no additional parts can be uploaded using that upload ID.                      |
| CreateMultipartUpload   | This action initiates a multipart upload and returns an upload ID. This upload ID is used to associate all of the parts in the specific multipart upload.  |
| CompleteMultipartUpload | Completes a multipart upload by assembling previously uploaded parts.                                                                                      |
| UploadPart              | Uploads a part in a multipart upload.                                                                                                                      |
| UploadPartCopy          | Uploads a part by copying data from an existing object as data source.                                                                                     |
| GetSignedUrl            | Supports pre-signed URLs for downloading and uploading objects.                                                                                            |

When a response payload is present, all responses are returned using UTF-8 encoded XML.

#### 2.17. Reference Documents <a href="#jutuh" id="jutuh"></a>

* [Download and install AWS CLI](https://aws.amazon.com/cli/)
* [AWS CLI Help](https://docs.aws.amazon.com/cli/latest/reference/s3/)
