Python web scrapping on GCP

Content

  1. build container images.
  2. tag images.
  3. upload images to GCP.
  4. create services.

build container images.

  • selenium.
  • web scrapping.

tag images.

  • image with GCP pattern.
docker pull selenium/standalone-chrome
# tag with pattern $GCP_REGISTRY/$PROJECT_NAME/$IMAGE_NAME
docker tag selenium_standalone-chrome asia.gcr.io/$PROJECT_NAME/selenium-standalone-chrome
# push image
docker push asia.gcr.io/$PROJECT_NAME/selenium-standalone-chrome

upload images to GCP.

  • authentication on browser.
gcloud auth login
  • setup project.
gcloud config set project $PROJECT_NAME

create services.

  • Create from container images.
# create selenium service.
gcloud run deploy selenium-chrome --image asia.gcr.io/$PROJECT_NAME/selenium_standalone-chrome --port 4444 --memory 2G --region asia-southeast1 --platform managed
  • Create from cloud build.
# gcloud builds submit --tag asia.gcr.io/$PROJECT_NAME/$IMAGE_NAME

gcloud builds submit --tag asia.gcr.io/$PROJECT_NAME/web-scrpping
# deploy service
gcloud run deploy selenium-chrome --image asia.gcr.io/$PROJECT_NAME/selenium_standalone-chrome

37