1. 简介
Eventarc 让您可以轻松地将各种服务(Cloud Run、Kubernetes、Workflows)与各种来源的事件连接起来。您可以用它来构建事件驱动型架构,其中微服务是松散耦合和分布式的。它还负责事件提取、传送、安全性、授权和错误处理,从而提高开发者的敏捷性和应用弹性。如需对 Eventarc 进行介绍,请参阅“使用 Eventarc 中的事件触发 Cloud Run”Codelab。
在此 Codelab 中,您将使用 Eventarc 从 Pub/Sub、Cloud Storage 和 Cloud Audit Logs 中读取事件,并将其传递给 Google Kubernetes Engine (GKE) 上运行的 Kubernetes 服务。
学习内容
- 创建 GKE 集群。
- 创建 GKE 服务作为事件接收器。
- 创建 Pub/Sub 触发器。
- 创建 Cloud Storage 触发器
- 创建 Cloud Audit Logs 触发器。
2. 设置和要求
自定进度的环境设置
- 登录 Google Cloud 控制台,然后创建一个新项目或重复使用现有项目。如果您还没有 Gmail 或 Google Workspace 账号,则必须创建一个。
- 项目名称是此项目参与者的显示名称。它是 Google API 尚未使用的字符串。您可以随时对其进行更新。
- 项目 ID 在所有 Google Cloud 项目中是唯一的,并且是不可变的(一经设置便无法更改)。Cloud 控制台会自动生成一个唯一字符串;通常您不在乎这是什么在大多数 Codelab 中,您都需要引用项目 ID(它通常标识为
PROJECT_ID
)。如果您不喜欢生成的 ID,可以再随机生成一个 ID。或者,您也可以尝试自己的项目 ID,看看是否可用。完成此步骤后便无法更改该 ID,并且该 ID 在项目期间会一直保留。 - 此外,还有第三个值,即某些 API 使用的项目编号,供您参考。如需详细了解所有这三个值,请参阅文档。
- 接下来,您需要在 Cloud 控制台中启用结算功能,以便使用 Cloud 资源/API。运行此 Codelab 应该不会产生太多的费用(如果有费用的话)。如需关停资源,以免产生超出本教程范围的结算费用,您可以删除自己创建的资源或删除整个项目。Google Cloud 的新用户符合参与 $300 USD 免费试用计划的条件。
启动 Cloud Shell
虽然可以通过笔记本电脑对 Google Cloud 进行远程操作,但在此 Codelab 中,您将使用 Google Cloud Shell,这是一个在云端运行的命令行环境。
在 Google Cloud 控制台 中,点击右上角工具栏中的 Cloud Shell 图标:
预配和连接到环境应该只需要片刻时间。完成后,您应该会看到如下内容:
这个虚拟机已加载了您需要的所有开发工具。它提供了一个持久的 5 GB 主目录,并且在 Google Cloud 中运行,大大增强了网络性能和身份验证功能。您在此 Codelab 中的所有工作都可以在浏览器中完成。您无需安装任何程序。
准备工作
在 Cloud Shell 中,确保项目 ID 已设置:
PROJECT_ID=your-project-id gcloud config set project $PROJECT_ID
3. 创建 GKE 集群
首先,为 GKE 启用所需的服务:
gcloud services enable container.googleapis.com
创建 Autopilot GKE 集群:
CLUSTER_NAME=eventarc-cluster REGION=us-central1 gcloud container clusters create-auto $CLUSTER_NAME --region $REGION
请确保已完成集群创建,然后再继续下一步。
4. 部署 GKE 服务
在部署服务之前,请先获取身份验证凭据,以便使用 kubectl
与集群进行交互:
gcloud container clusters get-credentials $CLUSTER_NAME \ --region $REGION
接下来,将 Cloud Run 的 hello 容器部署为 GKE 上的 Kubernetes 部署。此服务日志收到了 HTTP 请求和 CloudEvents:
SERVICE_NAME=hello-gke kubectl create deployment $SERVICE_NAME \ --image=gcr.io/cloudrun/hello
将 Deployment 作为内部 Kubernetes 服务公开。这将创建一个具有稳定 IP 的服务,可在集群内访问:
kubectl expose deployment $SERVICE_NAME \ --type ClusterIP --port 80 --target-port 8080
在继续下一步之前,请确保 Pod 正在运行:
kubectl get pods NAME READY STATUS hello-gke-df6469d4b-5vv22 1/1 Running
您还可以查看该服务:
kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP hello-gke LoadBalancer 10.51.1.26 <none>
5. 设置 Eventarc
在此步骤中,您将执行一些步骤,以设置 Eventarc 并初始化 Eventarc GKE 目标位置。
首先,为 Eventarc 和 Eventarc GKE 目标启用所需的服务:
gcloud services enable eventarc.googleapis.com \ cloudresourcemanager.googleapis.com
接下来,启用 Eventarc 以管理 GKE 集群:
gcloud eventarc gke-destinations init
Eventarc 会为每个针对 GKE 服务的触发器创建单独的事件转发器 Pod,并需要显式权限才能对集群进行更改。方法是向特殊服务账号授予管理集群中的资源的权限。您需要为每个 Google Cloud 项目执行一次此操作。
6. 活动发现
在创建触发器之前,您可以了解事件来源是什么、它们可以发出的事件类型,以及如何配置触发器以使用触发器。
您可以查看文档页面,了解 Eventarc 支持的事件。您还可以使用 gcloud
探索这些事件。
如需查看不同类型的事件的列表,请执行以下操作:
gcloud beta eventarc attributes types list NAME DESCRIPTION google.cloud.audit.log.v1.written Cloud Audit Log written google.cloud.pubsub.topic.v1.messagePublished Cloud Pub/Sub message published google.cloud.storage.object.v1.archived Cloud Storage: Sent when a live version of an (object versioned) object is archived or deleted. google.cloud.storage.object.v1.deleted Cloud Storage: Sent when an object has been permanently deleted. google.cloud.storage.object.v1.finalized Cloud Storage: Sent when a new object (or a new generation of an existing object). google.cloud.storage.object.v1.metadataUpdated Cloud Storage: Sent when the metadata of an existing object changes.
如需详细了解每种事件类型,请执行以下操作:
gcloud beta eventarc attributes types describe google.cloud.audit.log.v1.written attributes: type,serviceName,methodName,resourceName description: 'Cloud Audit Log: Sent when a log is written.' name: google.cloud.audit.log.v1.written
如需查看发出特定事件类型的服务列表,请执行以下操作:
gcloud beta eventarc attributes service-names list --type=google.cloud.audit.log.v1.written SERVICE_NAME DISPLAY_NAME accessapproval.googleapis.com Access Approval accesscontextmanager.googleapis.com Access Context Manager admin.googleapis.com Google Workspace Admin aiplatform.googleapis.com AI Platform (under Vertex AI) apigee.googleapis.com Apigee apigeeconnect.googleapis.com Apigee Connect ... workflows.googleapis.com Workflows
如需查看每个服务可以发出的方法名称(子事件)列表,请执行以下操作:
gcloud beta eventarc attributes method-names list --type=google.cloud.audit.log.v1.written --service-name=workflows.googleapis.com METHOD_NAME google.cloud.workflows.v1.Workflows.CreateWorkflow google.cloud.workflows.v1.Workflows.DeleteWorkflow google.cloud.workflows.v1.Workflows.GetWorkflow google.cloud.workflows.v1.Workflows.ListWorkflows google.cloud.workflows.v1.Workflows.UpdateWorkflow google.cloud.workflows.v1beta.Workflows.CreateWorkflow google.cloud.workflows.v1beta.Workflows.DeleteWorkflow google.cloud.workflows.v1beta.Workflows.GetWorkflow google.cloud.workflows.v1beta.Workflows.ListWorkflows google.cloud.workflows.v1beta.Workflows.UpdateWorkflow
7. 创建 Pub/Sub 触发器
接收事件的一种方法是通过 Pub/Sub。任何应用都可以将消息发布到 Pub/Sub,这些消息可通过 Eventarc 传送至服务。
设置
在创建任何触发器之前,您需要一个供触发器使用的服务账号。
创建服务账号:
SERVICE_ACCOUNT=eventarc-gke-trigger-sa gcloud iam service-accounts create $SERVICE_ACCOUNT
对于具有 GKE 目标位置的触发器,必须为服务账号授予以下角色:
roles/pubsub.subscriber
roles/monitoring.metricWriter
roles/eventarc.eventReceiver
(仅适用于 AuditLog 触发器。这将在后续步骤中添加)
授予角色:
gcloud projects add-iam-policy-binding $PROJECT_ID \ --member serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \ --role roles/pubsub.subscriber gcloud projects add-iam-policy-binding $PROJECT_ID \ --member serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \ --role roles/monitoring.metricWriter
创建
创建触发器以将 Pub/Sub 消息路由到您的服务:
TRIGGER_NAME=trigger-pubsub-gke gcloud eventarc triggers create $TRIGGER_NAME \ --destination-gke-cluster=$CLUSTER_NAME \ --destination-gke-location=$REGION \ --destination-gke-namespace=default \ --destination-gke-service=$SERVICE_NAME \ --destination-gke-path=/ \ --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \ --location=$REGION \ --service-account=$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com
测试
您可以通过列出所有触发器来检查是否已创建触发器:
gcloud eventarc triggers list
Pub/Sub 触发器会在后台创建 Pub/Sub 主题。让我们找到它并将其赋值给一个变量:
TOPIC_ID=$(gcloud eventarc triggers describe $TRIGGER_NAME --location $REGION --format='value(transport.pubsub.topic)')
使用 gcloud
向主题发布消息:
gcloud pubsub topics publish $TOPIC_ID --message="Hello World"
如需检查是否收到事件,首先请查找 Pod 名称:
kubectl get pods NAME READY STATUS hello-gke-df6469d4b-5vv22 1/1 Running
并将其存储到变量中:
POD_NAME=hello-gke-df6469d4b-5vv22
检查 Pod 的日志以验证收到的事件:
kubectl logs $POD_NAME { "severity": "INFO", "eventType": "google.cloud.pubsub.topic.v1.messagePublished", "message": "Received event of type google.cloud.pubsub.topic.v1.messagePublished. Event data: Hello World", "event": { "data": { "subscription": "projects/atamel-eventarc-gke/subscriptions/eventarc-us-central1-trigger-pubsub-gke-sub-270", "message": { "data": "SGVsbG8gV29ybGQ=", "messageId": "6031025573654834", "publishTime": "2022-10-19T14:13:07.990Z" } }, "datacontenttype": "application/json", "id": "6031025573654834", "source": "//pubsub.googleapis.com/projects/atamel-eventarc-gke/topics/eventarc-us-central1-trigger-pubsub-gke-729", "specversion": "1.0", "time": "2022-10-19T14:13:07.99Z", "type": "google.cloud.pubsub.topic.v1.messagePublished" } }
8. 创建 Cloud Storage 触发器
接收事件的另一种方法是通过 Cloud Storage。例如,应用可以将文件上传到存储分区,而该事件可以通过 Eventarc 传递给服务。
设置
在创建 Cloud Storage 触发器之前,请创建一个存储分区来接收以下来源的事件:
BUCKET_NAME=eventarc-gcs-$PROJECT_ID gsutil mb -l $REGION gs://$BUCKET_NAME
您还需要将 pubsub.publisher
角色添加到 Cloud Storage 触发器的 Cloud Storage 服务账号:
SERVICE_ACCOUNT_STORAGE=$(gsutil kms serviceaccount -p $PROJECT_ID) gcloud projects add-iam-policy-binding $PROJECT_ID \ --member serviceAccount:$SERVICE_ACCOUNT_STORAGE \ --role roles/pubsub.publisher
创建
创建一个触发器,以将新的文件创建事件从存储分区路由到您的服务:
TRIGGER_NAME=trigger-storage-gke gcloud eventarc triggers create $TRIGGER_NAME \ --destination-gke-cluster=$CLUSTER_NAME \ --destination-gke-location=$REGION \ --destination-gke-namespace=default \ --destination-gke-service=$SERVICE_NAME \ --destination-gke-path=/ \ --event-filters="type=google.cloud.storage.object.v1.finalized" \ --event-filters="bucket=$BUCKET_NAME" \ --location=$REGION \ --service-account=$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com
测试
您可以通过列出所有触发器来检查是否已创建触发器:
gcloud eventarc triggers list
创建一个文件,然后使用 gsutil
将文件上传到存储分区:
echo "Hello World" > random.txt gsutil cp random.txt gs://$BUCKET_NAME/random.txt
检查 Pod 的日志以验证收到的事件:
kubectl logs $POD_NAME { "severity": "INFO", "eventType": "google.cloud.storage.object.v1.finalized", "message": "Received event of type google.cloud.storage.object.v1.finalized. Event data: {\n \"kind\": \"storage#object\",\n \"id\": \"eventarc-gcs-atamel-eventarc-gke/random.txt/1666190425669022\",\n \"selfLink\": \"https://www.googleapis.com/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt\",\n \"name\": \"random.txt\",\n \"bucket\": \"eventarc-gcs-atamel-eventarc-gke\",\n \"generation\": \"1666190425669022\",\n \"metageneration\": \"1\",\n \"contentType\": \"text/plain\",\n \"timeCreated\": \"2022-10-19T14:40:25.678Z\",\n \"updated\": \"2022-10-19T14:40:25.678Z\",\n \"storageClass\": \"STANDARD\",\n \"timeStorageClassUpdated\": \"2022-10-19T14:40:25.678Z\",\n \"size\": \"12\",\n \"md5Hash\": \"5Z/5eUEET4XfUpfhwwLSYA==\",\n \"mediaLink\": \"https://storage.googleapis.com/download/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt?generation=1666190425669022\u0026alt=media\",\n \"contentLanguage\": \"en\",\n \"crc32c\": \"R1jUOQ==\",\n \"etag\": \"CJ77zIPD7PoCEAE=\"\n}\n", "event": { "bucket": "eventarc-gcs-atamel-eventarc-gke", "data": { "kind": "storage#object", "id": "eventarc-gcs-atamel-eventarc-gke/random.txt/1666190425669022", "selfLink": "https://www.googleapis.com/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt", "name": "random.txt", "bucket": "eventarc-gcs-atamel-eventarc-gke", "generation": "1666190425669022", "metageneration": "1", "contentType": "text/plain", "timeCreated": "2022-10-19T14:40:25.678Z", "updated": "2022-10-19T14:40:25.678Z", "storageClass": "STANDARD", "timeStorageClassUpdated": "2022-10-19T14:40:25.678Z", "size": "12", "md5Hash": "5Z/5eUEET4XfUpfhwwLSYA==", "mediaLink": "https://storage.googleapis.com/download/storage/v1/b/eventarc-gcs-atamel-eventarc-gke/o/random.txt?generation=1666190425669022\u0026alt=media", "contentLanguage": "en", "crc32c": "R1jUOQ==", "etag": "CJ77zIPD7PoCEAE=" }, "datacontenttype": "application/json", "id": "6031255652220627", "source": "//storage.googleapis.com/projects/_/buckets/eventarc-gcs-atamel-eventarc-gke", "specversion": "1.0", "subject": "objects/random.txt", "time": "2022-10-19T14:40:25.678152Z", "type": "google.cloud.storage.object.v1.finalized" } }
9. 创建 Cloud Audit Logs 触发器
虽然 Cloud Storage 触发器是监听 Cloud Storage 事件的更好方法,但在此步骤中,您将创建一个 Cloud Audit Logs 触发器来执行此操作。
设置
如需从服务接收事件,您需要启用审核日志。在 Google Cloud 控制台中,选择左上角的 IAM & Admin
和 Audit Logs
。在服务列表中,选中 Google Cloud Storage
:
在右侧,确保选中 Admin
、Read
和 Write
,然后点击 Save
:
您还需要将 eventarc.eventReceiver
角色添加到 Cloud Audit Logs 触发器的触发器服务账号:
gcloud projects add-iam-policy-binding $PROJECT_ID \ --member serviceAccount:$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com \ --role roles/eventarc.eventReceiver
创建
创建一个触发器,以将新的文件创建事件从存储分区路由到您的服务:
TRIGGER_NAME=trigger-auditlog-storage-gke gcloud eventarc triggers create $TRIGGER_NAME \ --destination-gke-cluster=$CLUSTER_NAME \ --destination-gke-location=$REGION \ --destination-gke-namespace=default \ --destination-gke-service=$SERVICE_NAME \ --destination-gke-path=/ \ --event-filters="type=google.cloud.audit.log.v1.written" \ --event-filters="serviceName=storage.googleapis.com" \ --event-filters="methodName=storage.objects.create" \ --event-filters-path-pattern="resourceName=/projects/_/buckets/$BUCKET_NAME/objects/*" \ --location=$REGION \ --service-account=$SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com
测试
审核日志触发器需要一些时间进行初始化。您可以通过列出所有触发器来检查是否已创建触发器:
gcloud eventarc triggers list
您应该会看到 Active
字段为 Yes
:
NAME TYPE DESTINATION ACTIVE trigger-auditlog-storage-gke google.cloud.audit.log.v1.written GKE: hello-gke Yes
创建一个文件,然后使用 gsutil
将文件上传到存储分区:
echo "Hello World" > random.txt gsutil cp random.txt gs://$BUCKET_NAME/random.txt
检查 Pod 的日志以验证收到的事件:
kubectl logs $POD_NAME { "severity": "INFO", "eventType": "google.cloud.audit.log.v1.written", "message": "Received event of type google.cloud.audit.log.v1.written. Event data: {\"protoPayload\":{\"status\":{},\"authenticationInfo\":{\"principalEmail\":\"atameldev@gmail.com\"},\"requestMetadata\":{\"callerIp\":\"149.71.143.227\",\"callerSuppliedUserAgent\":\"apitools Python/3.10.4 gsutil/5.14 (darwin) analytics/disabled interactive/True command/cp google-cloud-sdk/405.0.1,gzip(gfe)\",\"requestAttributes\":{\"time\":\"2022-10-19T15:05:54.144615670Z\",\"auth\":{}},\"destinationAttributes\":{}},\"serviceName\":\"storage.googleapis.com\",\"methodName\":\"storage.objects.create\",\"authorizationInfo\":[{\"resource\":\"projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt\",\"permission\":\"storage.objects.delete\",\"granted\":true,\"resourceAttributes\":{}},{\"resource\":\"projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt\",\"permission\":\"storage.objects.create\",\"granted\":true,\"resourceAttributes\":{}}],\"resourceName\":\"projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt\",\"serviceData\":{\"@type\":\"type.googleapis.com/google.iam.v1.logging.AuditData\",\"policyDelta\":{\"bindingDeltas\":[{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectOwner\",\"member\":\"projectOwner:atamel-eventarc-gke\"},{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectOwner\",\"member\":\"projectEditor:atamel-eventarc-gke\"},{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectOwner\",\"member\":\"user:atameldev@gmail.com\"},{\"action\":\"ADD\",\"role\":\"roles/storage.legacyObjectReader\",\"member\":\"projectViewer:atamel-eventarc-gke\"}]}},\"resourceLocation\":{\"currentLocations\":[\"us-central1\"]}},\"insertId\":\"-8vmrbve7pol2\",\"resource\":{\"type\":\"gcs_bucket\",\"labels\":{\"project_id\":\"atamel-eventarc-gke\",\"bucket_name\":\"eventarc-gcs-atamel-eventarc-gke\",\"location\":\"us-central1\"}},\"timestamp\":\"2022-10-19T15:05:54.138732321Z\",\"severity\":\"INFO\",\"logName\":\"projects/atamel-eventarc-gke/logs/cloudaudit.googleapis.com%2Fdata_access\",\"receiveTimestamp\":\"2022-10-19T15:05:54.839604461Z\"}", "event": { "data": { "protoPayload": { "status": { }, "authenticationInfo": { "principalEmail": "atameldev@gmail.com" }, "requestMetadata": { "callerIp": "149.71.143.227", "callerSuppliedUserAgent": "apitools Python/3.10.4 gsutil/5.14 (darwin) analytics/disabled interactive/True command/cp google-cloud-sdk/405.0.1,gzip(gfe)", "requestAttributes": { "time": "2022-10-19T15:05:54.144615670Z", "auth": { } }, "destinationAttributes": { } }, "serviceName": "storage.googleapis.com", "methodName": "storage.objects.create", "authorizationInfo": [ { "resource": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt", "permission": "storage.objects.delete", "granted": true, "resourceAttributes": { } }, { "resource": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt", "permission": "storage.objects.create", "granted": true, "resourceAttributes": { } } ], "resourceName": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt", "serviceData": { "@type": "type.googleapis.com/google.iam.v1.logging.AuditData", "policyDelta": { "bindingDeltas": [ { "action": "ADD", "role": "roles/storage.legacyObjectOwner", "member": "projectOwner:atamel-eventarc-gke" }, { "action": "ADD", "role": "roles/storage.legacyObjectOwner", "member": "projectEditor:atamel-eventarc-gke" }, { "action": "ADD", "role": "roles/storage.legacyObjectOwner", "member": "user:atameldev@gmail.com" }, { "action": "ADD", "role": "roles/storage.legacyObjectReader", "member": "projectViewer:atamel-eventarc-gke" } ] } }, "resourceLocation": { "currentLocations": [ "us-central1" ] } }, "insertId": "-8vmrbve7pol2", "resource": { "type": "gcs_bucket", "labels": { "project_id": "atamel-eventarc-gke", "bucket_name": "eventarc-gcs-atamel-eventarc-gke", "location": "us-central1" } }, "timestamp": "2022-10-19T15:05:54.138732321Z", "severity": "INFO", "logName": "projects/atamel-eventarc-gke/logs/cloudaudit.googleapis.com%2Fdata_access", "receiveTimestamp": "2022-10-19T15:05:54.839604461Z" }, "datacontenttype": "application/json; charset=utf-8", "dataschema": "https://googleapis.github.io/google-cloudevents/jsonschema/google/events/cloud/audit/v1/LogEntryData.json", "id": "projects/atamel-eventarc-gke/logs/cloudaudit.googleapis.com%2Fdata_access-8vmrbve7pol21666191954138732", "methodname": "storage.objects.create", "recordedtime": "2022-10-19T15:05:54.138732321Z", "resourcename": "projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt", "servicename": "storage.googleapis.com", "source": "//cloudaudit.googleapis.com/projects/atamel-eventarc-gke/logs/data_access", "specversion": "1.0", "subject": "storage.googleapis.com/projects/_/buckets/eventarc-gcs-atamel-eventarc-gke/objects/random.txt", "time": "2022-10-19T15:05:54.839604461Z", "type": "google.cloud.audit.log.v1.written" } }
10. 恭喜!
恭喜您完成此 Codelab。
所学内容
- 创建 GKE 集群。
- 创建 GKE 服务作为事件接收器。
- 创建 Pub/Sub 触发器。
- 创建 Cloud Storage 触发器。
- 创建 Cloud Audit Logs 触发器。