Data Transfer Service in GCP


Data transfer from other Cloud Providers 


This is an automated way of handling the data transfer between the GCS buckets, other storage solutions like AWS S3 or any HTTPS server. We can use this service for transferring the data across the cloud as well as moving from on-premise to cloud. 

We can schedule the data transfer job and also filter specific data that will be imported. For typical data transfer between on-premise to cloud or transfer across the buckets, gsutil is an appropriate option. If we want to move data from AWS/Azure/HTTPS servers to GCP, then this service will be a recommended approach. 

Go to the storage, data transfer service and create the job. 

If we transfer the data from S3, select the bucket along with access and the secret key. 


Make sure to have appropriate permission to access the files from GCS,

data transfer service in GCP

datacloudies
Once the job is complete, verify the object in target,

gcs bucket





If we want to transfer the data from another bucket, go ahead with below options

data transfer service

There are additional filters where we can include/exclude certain files based on the prefixes and also by the last-modified time range. 

Select the destination bucket with transfer options as per the requirement,


Run the job now,

data transfer service in GCP




Once the job triggered, wait for the status turns to complete state,

data transfer service in GCP

Verify the copied objects in the target bucket,

gcs bucket



That is it. I hope this is clear. Please do follow the next posts for more readings. Thank you.


Recent Posts

  • Cloud SQL
    What is Cloud SQL  It is a fully managed relational database service for PostgreSQL and MySQL database in the GCP. MySQL instance can be...
    May 17 2020 | Read more
  • Deploy an application to Google Kubernetes Engine
    Welcome back, this is the continuation post of the previous article, where we have discussed the Kubernetes and its concepts in detail. In this...
    May 13 2020 | Read more
  • Google Kubernetes Engine
    What is GKE? GKE is a managed, production-ready environment for deploying containerized applications. It is like a managed cargo container in...
    May 13 2020 | Read more
  • Google App Engine – Flexible
    In the previous article, we have discussed what is google app engine and how to deploy the application in the Google app engine – Standard. In...
    May 10 2020 | Read more