Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X............X..........................................] duration:
920.175215ms
FAIL (2025-08-09 04:10:52 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 920.204528ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [...X...................X..........X.........................] duration:
1.067233721s
FAIL (2025-08-09 04:10:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.067264033s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [....X...................X..........X........................] duration:
1.062666473s
FAIL (2025-08-09 04:10:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.062692562s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X............X.........................................] duration:
848.469162ms
FAIL (2025-08-09 04:10:33 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 848.50559ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [..................X..XXX.....X.X............................] duration:
331.858053ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:10:29 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 331.880237ms
edge
gateway
vhost
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
4 out of 60 is failed (threshold=3) [.....X........................................X.X......X....] duration:
850.04845ms
FAIL (2025-08-09 04:10:29 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 850.077798ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X.......................X..................] duration:
2.432784804s
FAIL (2025-08-09 04:10:10 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.432812466s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X............X........................................] duration:
1.140136899s
FAIL (2025-08-09 04:09:55 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.140164785s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X.......................X.................] duration:
2.942690587s
FAIL (2025-08-09 04:09:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.942718288s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [....................X.........X............X................] duration:
3.890834788s
FAIL (2025-08-09 04:09:40 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 3.890865538s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [.....X...................X..........X.......................] duration:
1.035143732s
FAIL (2025-08-09 04:09:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.035167036s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
4 out of 60 is failed (threshold=3) [......X........................................X.X......X...] duration:
340.777637ms
FAIL (2025-08-09 04:09:35 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 340.816548ms
edge
authservice
http
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [...................X..XXX.....X.X...........................] duration:
340.324629ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:09:33 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 340.33919ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......X............X.......................................] duration:
840.354873ms
FAIL (2025-08-09 04:09:26 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 840.386805ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [......X...................X..........X......................] duration:
1.013535069s
FAIL (2025-08-09 04:08:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.013562908s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [X....................X.........X............X...............] duration:
2.711899571s
FAIL (2025-08-09 04:08:44 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 2.711927968s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X.......................X................] duration:
2.546788287s
FAIL (2025-08-09 04:08:41 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.546817342s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........X............X......................................] duration:
925.169566ms
FAIL (2025-08-09 04:08:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 925.200703ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [....................X..XXX.....X.X..........................] duration:
302.036546ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:08:37 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 302.051769ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.X....................X.........X............X..............] duration:
2.749848119s
FAIL (2025-08-09 04:08:25 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 2.750081376s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
4 out of 60 is failed (threshold=3) [.......X........................................X.X......X..] duration:
363.692455ms
FAIL (2025-08-09 04:08:21 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 363.716796ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X.......................X...............] duration:
2.959454904s
FAIL (2025-08-09 04:08:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.959489445s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [..X....................X.........X............X.............] duration:
3.039685713s
FAIL (2025-08-09 04:07:52 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 3.039708869s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [.....................X..XXX.....X.X.........................] duration:
297.36092ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:07:43 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 297.387744ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......X...................X..........X.....................] duration:
2.284569787s
FAIL (2025-08-09 04:07:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.284596344s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........X............X.....................................] duration:
897.663321ms
FAIL (2025-08-09 04:07:32 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 897.6895ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
4 out of 60 is failed (threshold=3) [........X........................................X.X......X.] duration:
319.886512ms
FAIL (2025-08-09 04:07:22 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 319.905883ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [........X...................X..........X....................] duration:
1.124216226s
FAIL (2025-08-09 04:07:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.124242214s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [......................X..XXX.....X.X........................] duration:
292.070614ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:06:59 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 292.098845ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.......................X..............] duration:
2.368771622s
FAIL (2025-08-09 04:06:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.368803388s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........X............X....................................] duration:
894.249746ms
FAIL (2025-08-09 04:06:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 894.276598ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [...X....................X.........X............X............] duration:
2.534550338s
FAIL (2025-08-09 04:06:39 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 2.53458724s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................X.......................X.............] duration:
2.345842514s
FAIL (2025-08-09 04:06:35 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.345870281s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........X...................X..........X...................] duration:
1.184280407s
FAIL (2025-08-09 04:06:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.184309273s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [....X....................X.........X............X...........] duration:
2.900381879s
FAIL (2025-08-09 04:06:05 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 2.900452849s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [.......................X..XXX.....X.X.......................] duration:
308.868258ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:06:04 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 308.886777ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.......................X............] duration:
2.34359631s
FAIL (2025-08-09 04:05:49 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.343625047s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [..........X...................X..........X..................] duration:
1.036110616s
FAIL (2025-08-09 04:05:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.036136187s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [........................X..XXX.....X.X......................] duration:
339.290352ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:05:33 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 339.313078ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........X............X...................................] duration:
890.508205ms
FAIL (2025-08-09 04:05:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 890.562383ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [...........X...................X..........X.................] duration:
1.11468994s
FAIL (2025-08-09 04:05:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.114714183s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [.....X....................X.........X............X..........] duration:
2.781276534s
FAIL (2025-08-09 04:05:21 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 2.78130733s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............X............X..................................] duration:
875.126666ms
FAIL (2025-08-09 04:05:07 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 875.15756ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X.......................X...........] duration:
2.490702895s
FAIL (2025-08-09 04:04:52 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.490734599s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [............X...................X..........X................] duration:
1.118599563s
FAIL (2025-08-09 04:04:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.118625983s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [......X....................X.........X............X.........] duration:
2.94493815s
FAIL (2025-08-09 04:04:37 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 2.94519897s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
3 out of 60 is failed (threshold=3) [..........X........................................X.X......] duration:
324.231997ms
FAIL (2025-08-09 04:04:26 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 324.268535ms
edge
authservice
http
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
6 out of 60 is failed (threshold=3) [.........................X..XXX.....X.X.....................] duration:
333.949378ms remote:
"tcp:45.38.24.2:443" target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-09 04:04:23 UTC)
orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded
Duration: 333.977671ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................X.......................X..........] duration:
2.444711968s
FAIL (2025-08-09 04:04:01 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 9 03:55:22.143759676 (10h29m44.10285692s) [upload=10871]: Starting upload
Aug 9 03:55:22.143812813 (10h29m44.102910057s) [upload=10871]: Done starting upload
Aug 9 03:55:22.143932181 (10h29m44.103029427s) [upload=10871]: Uploading segments...
Aug 9 03:55:22.144063992 (10h29m44.103161237s) [upload=10871]: Got next segment. Inline: false
Aug 9 03:55:22.144103663 (10h29m44.103200909s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 03:55:22.144108033 (10h29m44.103205279s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 9 03:55:22.344140014 (10h29m44.303237259s) [seg_pos={0 0}, upload=10871]: Begin upload segment...
Aug 9 03:55:22.344190230 (10h29m44.303287465s) [seg_pos={0 0}, upload=10871]: Done begin upload segment.
Aug 9 03:55:22.344241693 (10h29m44.303338943s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.344712874 (10h29m44.303810123s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345037423 (10h29m44.304134668s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345359148 (10h29m44.304456399s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.345744398 (10h29m44.304841644s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346149512 (10h29m44.305246803s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346393756 (10h29m44.305491013s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346552983 (10h29m44.305650229s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346804907 (10h29m44.30590216s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.346863880 (10h29m44.305961132s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347094395 (10h29m44.306191643s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347154723 (10h29m44.306251977s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347352448 (10h29m44.306449697s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347443699 (10h29m44.306540953s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.347752140 (10h29m44.30684939s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348045599 (10h29m44.307142858s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348372798 (10h29m44.307470046s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.348673074 (10h29m44.30777033s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Uploading piece...
Aug 9 03:55:22.911067156 (10h29m44.870164401s) [upload=10871]: Next returned nil segment
Aug 9 03:55:22.911084172 (10h29m44.870181416s) [upload=10871]: Waiting for error group managing segments...
Aug 9 03:55:23.048261951 (10h29m45.007359196s) [noise=true , address=45.91.116.14:4200 , piece_id=3NK2RKJPXS4UAJXY55I2LHQRX7D3ZPKWFG3SMSV4EZ7MHUROSGRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.064939046 (10h29m45.024036292s) [noise=true , address=104.36.82.60:4359 , piece_id=2DZPFT7KW2BGOQZXKLQM64FS4Q7QJQPQUNTDMRPVKV5JU3KQN4MQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.066143838 (10h29m45.025241084s) [noise=true , address=104.36.82.47:4533 , piece_id=SR7G6S2Z3AGRWND572QYBPMKDNRK3MD6QRUMBDYLRM2ZYVNFVUTQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.068805794 (10h29m45.027903038s) [noise=true , address=104.36.82.59:4323 , piece_id=YVKZVOU4QTUNNHC3W3UUF3KISTPJ3HMU5QIGGFOPFRLS2R3XEF3Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.074735109 (10h29m45.03383234s) [noise=true , address=104.36.82.45:4913 , piece_id=A53X5FREON2B3TPU4OJA6GY7743MG63MPW6KJEQ4ICMJEV5QZMYQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.076585738 (10h29m45.035682976s) [noise=true , address=104.36.82.15:4267 , piece_id=6QVBPNZGKNZEH35LRTWXLMZIVNYSUXFLGF4UXA7VOBUUI4LHRSLQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.078307083 (10h29m45.037404316s) [noise=true , address=104.36.82.44:4872 , piece_id=ZDQMVJGYPL4R7H2MSYDPVGLKN4B7FG3SCGVZMDU5MFLVISQ3DC6Q, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.080903255 (10h29m45.040000501s) [noise=true , address=104.36.82.58:4296 , piece_id=AGVDDRLZXAAXATA72TY76JNH7P6J6RSFRZFCI7FNT6SSEBNQ6HAA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.084965825 (10h29m45.044063057s) [noise=true , address=104.36.82.2:4759 , piece_id=B35NJJJ7E6ULFD5OGVDMH4OIC77IGELIRP7AUAJWOYJAGKPDNWRA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.086897888 (10h29m45.045995124s) [noise=true , address=104.36.82.45:4907 , piece_id=BP2W2L76MWDS6W2PQRPK2P3COOD75G75D37ZUGFCY2NOWXBSZGEQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.088234355 (10h29m45.047331591s) [noise=true , address=104.36.82.46:4482 , piece_id=MJM7TS3NQTQJ2D4SMYDGDS2FX3AUICTMXSYJO2LE4F3F74CFBOAQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.090326389 (10h29m45.049423623s) [noise=true , address=104.36.82.12:4158 , piece_id=NDGI2BLPE4IMV2JWEZ5AXPHRBT6MSME4UNNGYDOKDOIA6PYSS3TA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096118399 (10h29m45.055215633s) [noise=true , address=104.36.82.23:4458 , piece_id=5R5O366APTWEUDTZG5GWTSDBNX5S53VZPO2K2U3ZD45OEXBSPHUA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096577595 (10h29m45.05567483s) [noise=true , address=104.36.82.38:4720 , piece_id=6BMWQEPTWJXMPD5VYOCHEERBK374TUXUNUFEH4W427CGZU33Z4CA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: <nil>
Aug 9 03:55:23.096630675 (10h29m45.05572792s) [seg_pos={0 0}, upload=10871]: Segment reached optimal threshold of 14 pieces.
Aug 9 03:55:23.096843965 (10h29m45.055941203s) [noise=true , address=104.36.82.21:4496 , piece_id=OPJLKLLJ7SUYZ4BZ325AANRWXDQJM6JPBSC6JHT3WTI5SGVRHQNA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1uPsj1cL28HYcNZYjinFAwBHx4bWeSz1E1ndTDYE3y6DwbZg7w): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097165055 (10h29m45.056262294s) [noise=true , address=104.36.82.16:4319 , piece_id=JZKTOUWZW7SWGC3737OGKA5R2V64ANU5ZIXNMLDA4GKCG77Y55HA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hQTG2DzecaoN1Na2mY3JJqNj2qxtJDQkhMYmRmd9hve8D7T9C): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097248518 (10h29m45.056345754s) [noise=true , address=64.49.13.140:4511 , piece_id=T7H2CXRTLLZOTJTPVD6BYSVSPWEGB4Y3C2VCIZMZGX5F3RKRYNLA, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F79nPRFzy9hjCq5KYvLy6bZERw9cEvczCN2o66Cf6ccKEpkAk): protocol: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59802->64.49.13.140:4511: use of closed network connection
Aug 9 03:55:23.097330100 (10h29m45.056427345s) [noise=true , address=104.36.82.39:4739 , piece_id=R3OAUKXTNBEEZMXX3CJDT2CASE3HLBXNPKD4EGRRFFFF7EGTQ2YQ, seg_pos={0 0}, upload=10871]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A4nWQsS43tSv3atCnj6CB2ZGKFxQUHJp8a7ftP9y1XumeFk33): protocol: expected piece hash; context canceled; EOF
Aug 9 03:55:23.097345526 (10h29m45.056442772s) [seg_pos={0 0}, upload=10871]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 9 03:55:23.097442993 (10h29m45.056540238s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 9 03:55:23.097448165 (10h29m45.056545413s) [upload=10871]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 03:55:23.258254104 (10h29m45.21735135s) [upload=10871]: Done uploading segments...
Aug 9 03:55:23.258294964 (10h29m45.217392209s) [upload=10871]: Upload finished. err: <nil>
Duration: 2.444799542s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............X............X.................................] duration:
989.954807ms
FAIL (2025-08-09 04:03:56 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 03:38:21.391404361 (10h12m43.350501607s) [upload=10580]: Starting upload Aug 9 03:38:21.391499574 (10h12m43.350596824s) [upload=10580]: Done starting upload Aug 9 03:38:21.391580214 (10h12m43.35067745s) [upload=10580]: Uploading segments... Aug 9 03:38:21.391626103 (10h12m43.350723357s) [upload=10580]: Got next segment. Inline: true Aug 9 03:38:21.391640756 (10h12m43.350738001s) [upload=10580]: Waiting for error group managing segments... Aug 9 03:38:21.391684974 (10h12m43.350782221s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 03:38:21.391714671 (10h12m43.350811918s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 9 03:38:21.391718801 (10h12m43.35081605s) [upload=10580]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 9 03:38:21.563255973 (10h12m43.522353225s) [upload=10580]: Done uploading segments... Aug 9 03:38:21.563291106 (10h12m43.522388342s) [upload=10580]: Upload finished. err: metaclient: Too Many Requests
Duration: 990.001227ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|