[22:30:44.046] New invocation is queued and will start shortly
[22:30:45.108] Starting the invocation (attempt 1)
[22:30:45.133] Ignoring gitiles_ref tag without the buildset tag
[22:30:45.133] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:30:45.135] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:30:45.135] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:31:21.134] Failed to schedule Buildbucket build - rpc error: code = Internal desc =
[22:31:21.134] The invocation will be retried
[22:31:26.208] Starting the invocation (attempt 2)
[22:31:26.235] Ignoring gitiles_ref tag without the buildset tag
[22:31:26.235] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:31:26.235] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:31:26.236] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:32:03.764] Failed to schedule Buildbucket build - rpc error: code = Internal desc =
[22:32:03.764] The invocation will be retried
[22:32:13.818] Starting the invocation (attempt 3)
[22:32:13.847] Ignoring gitiles_ref tag without the buildset tag
[22:32:13.847] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:32:13.848] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:32:13.848] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:32:54.144] Failed to schedule Buildbucket build - rpc error: code = Internal desc =
[22:32:54.144] The invocation will be retried
[22:33:14.193] Starting the invocation (attempt 4)
[22:33:14.219] Ignoring gitiles_ref tag without the buildset tag
[22:33:14.219] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:33:14.219] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:33:14.219] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:33:48.582] Failed to schedule Buildbucket build - rpc error: code = Internal desc =
[22:33:48.582] The invocation will be retried
[22:34:29.330] Starting the invocation (attempt 5)
[22:34:29.365] Ignoring gitiles_ref tag without the buildset tag
[22:34:29.365] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:34:29.365] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:34:29.365] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:35:03.339] Failed to schedule Buildbucket build - rpc error: code = Internal desc = 502 Bad Gateway nginx
[22:35:03.339] The invocation will be retried
[22:36:23.397] Starting the invocation (attempt 6)
[22:36:23.422] Ignoring gitiles_ref tag without the buildset tag
[22:36:23.422] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:36:23.422] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:36:23.422] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:36:57.429] Failed to schedule Buildbucket build - rpc error: code = Internal desc =
[22:36:57.429] The invocation will be retried
[22:39:38.109] Starting the invocation (attempt 7)
[22:39:38.138] Ignoring gitiles_ref tag without the buildset tag
[22:39:38.138] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:39:38.139] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:39:38.139] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:40:13.548] Failed to schedule Buildbucket build - rpc error: code = Internal desc = 502 Bad Gateway nginx
[22:40:13.548] The invocation will be retried
[22:45:33.605] Starting the invocation (attempt 8)
[22:45:33.637] Ignoring gitiles_ref tag without the buildset tag
[22:45:33.637] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:45:33.638] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:45:33.638] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:46:07.372] Failed to schedule Buildbucket build - rpc error: code = Internal desc =
[22:46:07.372] The invocation will be retried
[22:56:07.414] Starting the invocation (attempt 9)
[22:56:07.440] Ignoring gitiles_ref tag without the buildset tag
[22:56:07.440] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[22:56:07.442] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[22:56:07.442] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[22:56:42.418] Failed to schedule Buildbucket build - rpc error: code = Internal desc = 502 Bad Gateway nginx
[22:56:42.418] The invocation will be retried
[23:06:42.521] Starting the invocation (attempt 10)
[23:06:42.546] Ignoring gitiles_ref tag without the buildset tag
[23:06:42.546] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[23:06:42.548] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[23:06:42.548] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[23:07:16.759] Failed to schedule Buildbucket build - rpc error: code = Internal desc = 502 Bad Gateway nginx
[23:07:16.760] The invocation will be retried
[23:17:16.827] Starting the invocation (attempt 11)
[23:17:16.857] Ignoring gitiles_ref tag without the buildset tag
[23:17:16.857] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[23:17:16.860] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[23:17:16.860] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[23:17:51.688] Failed to schedule Buildbucket build - rpc error: code = Internal desc =
[23:17:51.688] The invocation will be retried
[23:27:52.124] Starting the invocation (attempt 12)
[23:27:52.151] Ignoring gitiles_ref tag without the buildset tag
[23:27:52.151] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[23:27:52.225] Failed to prepare PubSub topic - auth: HTTP code (502) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[23:27:52.225] The invocation will be retried
[23:37:52.274] Starting the invocation (attempt 13)
[23:37:52.302] Ignoring gitiles_ref tag without the buildset tag
[23:37:52.302] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[23:37:52.532] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[23:37:52.532] The invocation will be retried
[23:47:52.577] Starting the invocation (attempt 14)
[23:47:52.605] Ignoring gitiles_ref tag without the buildset tag
[23:47:52.605] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[23:47:53.389] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[23:47:53.389] The invocation will be retried
[23:57:54.186] Starting the invocation (attempt 15)
[23:57:54.211] Ignoring gitiles_ref tag without the buildset tag
[23:57:54.211] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[23:57:54.972] Failed to prepare PubSub topic - auth: HTTP code (502) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[23:57:54.972] The invocation will be retried
[00:07:55.025] Starting the invocation (attempt 16)
[00:07:55.059] Ignoring gitiles_ref tag without the buildset tag
[00:07:55.059] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[00:07:55.684] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[00:07:55.684] The invocation will be retried
[00:17:55.730] Starting the invocation (attempt 17)
[00:17:55.756] Ignoring gitiles_ref tag without the buildset tag
[00:17:55.756] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[00:17:56.783] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[00:17:56.783] The invocation will be retried
[00:27:56.842] Starting the invocation (attempt 18)
[00:27:56.880] Ignoring gitiles_ref tag without the buildset tag
[00:27:56.880] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[00:27:57.797] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[00:27:57.797] The invocation will be retried
[00:37:57.852] Starting the invocation (attempt 19)
[00:37:57.876] Ignoring gitiles_ref tag without the buildset tag
[00:37:57.876] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[00:37:58.959] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[00:37:58.959] The invocation will be retried
[00:47:59.006] Starting the invocation (attempt 20)
[00:47:59.030] Ignoring gitiles_ref tag without the buildset tag
[00:47:59.030] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[00:47:59.988] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[00:47:59.988] The invocation will be retried
[00:58:00.035] Starting the invocation (attempt 21)
[00:58:00.059] Ignoring gitiles_ref tag without the buildset tag
[00:58:00.060] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[00:58:00.184] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[00:58:00.185] The invocation will be retried
[01:08:00.238] Starting the invocation (attempt 22)
[01:08:00.264] Ignoring gitiles_ref tag without the buildset tag
[01:08:00.264] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[01:08:00.862] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[01:08:00.862] The invocation will be retried
[01:18:00.915] Starting the invocation (attempt 23)
[01:18:00.943] Ignoring gitiles_ref tag without the buildset tag
[01:18:00.943] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[01:18:01.032] Failed to prepare PubSub topic - auth: HTTP code (502) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[01:18:01.032] The invocation will be retried
[01:28:01.090] Starting the invocation (attempt 24)
[01:28:01.125] Ignoring gitiles_ref tag without the buildset tag
[01:28:01.125] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[01:28:01.347] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[01:28:01.347] The invocation will be retried
[01:38:01.401] Starting the invocation (attempt 25)
[01:38:01.429] Ignoring gitiles_ref tag without the buildset tag
[01:38:01.429] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[01:38:01.873] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[01:38:01.873] The invocation will be retried
[01:48:01.923] Starting the invocation (attempt 26)
[01:48:01.946] Ignoring gitiles_ref tag without the buildset tag
[01:48:01.946] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[01:48:03.072] Failed to prepare PubSub topic - auth: HTTP code (500) when fetching https://cr-buildbucket-dev.appspot.com/auth/api/v1/server/info
[01:48:03.072] The invocation will be retried
[01:58:03.125] Starting the invocation (attempt 27)
[01:58:03.150] Ignoring gitiles_ref tag without the buildset tag
[01:58:03.150] Preparing PubSub topic for "https://cr-buildbucket-dev.appspot.com"
[01:58:03.150] PubSub topic is "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
[01:58:03.150] Buildbucket request:
{
"requestId": "8860174414689357376",
"builder": {
"project": "infra",
"bucket": "ci",
"builder": "build-proto-linux"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler-dev.appspot.com",
"invocation": "8860174414689357376",
"job": "infra/build-proto-linux",
"triggers": [
{
"cron": {
"generation": "1493974"
},
"id": "cron:v1:1493974"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860174414689357376"
},
{
"key": "scheduler_job_id",
"value": "infra/build-proto-linux"
},
{
"key": "user_agent",
"value": "luci-scheduler-dev"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler-dev/topics/scheduler.buildbucket.cr-buildbucket-dev~appspot.gserviceaccount.com"
}
}
[01:58:03.775] Received PubSub notification, asking Buildbucket for the build status
[01:58:03.775] Fatal error when handling PubSub notification, aborting invocation - unexpected invocation status "RETRYING", expecting "RUNNING"
[01:58:03.775] Invocation finished in 3h27m19.73742526s with status FAILED