Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.323091433s FAIL (2025-07-25 10:10:19 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.323131493s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 336.303312ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Request-Id=DBm2nRVQhvj Accept-Ranges=bytes Content-Length=279 Server=Storj X-Amz-Request-Id=185574E527952D7A X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:16 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:10:11 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E527952D7A</RequestId><HostId></HostId></Error>" Duration: 336.332002ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............XX...............................X..............] duration: 1.055939675s FAIL (2025-07-25 10:10:09 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 1.056020469s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 1.095057094s FAIL (2025-07-25 10:10:03 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.095088219s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.301329426s FAIL (2025-07-25 10:09:57 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 21h59m10.08413493s) [seg_pos={0 0}, upload=21166]: Done begin upload segment. Jul 25 09:34:38.893861575 (21h59m10.084160157s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893877564 (21h59m10.084176144s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893970156 (21h59m10.08426874s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894325201 (21h59m10.084623773s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894381124 (21h59m10.084679703s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894891519 (21h59m10.085190109s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895339175 (21h59m10.085637757s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895721362 (21h59m10.086019941s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896080111 (21h59m10.086378691s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896408696 (21h59m10.086707281s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896758554 (21h59m10.087057132s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897033623 (21h59m10.087332202s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897322453 (21h59m10.087621038s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897596601 (21h59m10.087895181s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897880857 (21h59m10.088179437s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898303843 (21h59m10.088602412s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898661671 (21h59m10.08896025s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.899092762 (21h59m10.089391347s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:39.465168779 (21h59m10.655467365s) [upload=21166]: Next returned nil segment Jul 25 09:34:39.465194393 (21h59m10.655492983s) [upload=21166]: Waiting for error group managing segments... Jul 25 09:34:39.598220252 (21h59m10.788518839s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.604836354 (21h59m10.795134975s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.612452239 (21h59m10.80275081s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.619515598 (21h59m10.809814179s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.626845628 (21h59m10.817144209s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.627851431 (21h59m10.818150011s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.629266997 (21h59m10.819565577s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.645882287 (21h59m10.836180867s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.646651873 (21h59m10.836950453s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.647387132 (21h59m10.837685716s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.653519594 (21h59m10.843818177s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.660641895 (21h59m10.850940478s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.711883970 (21h59m10.902182542s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761622344 (21h59m10.951920924s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761651494 (21h59m10.951950073s) [seg_pos={0 0}, upload=21166]: Segment reached optimal threshold of 14 pieces. Jul 25 09:34:39.762107894 (21h59m10.952406474s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:39.762263719 (21h59m10.952562302s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qfp6gac6HdZquSvy2ouWCkMKieFDAN4JUQXg1zHSD3bNCBveV): protocol: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection Jul 25 09:34:39.762288380 (21h59m10.952586956s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TLFJ7HD7rVyLCyaKdy7DAJ1XvdN6s6gg6FuKQ58oJsRV6fhU6): protocol: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection Jul 25 09:34:39.762335180 (21h59m10.952633759s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vyNSWnc7efrHbDqsTmoeuMqg5JrS9dCA1sxEK4J3SfgJV5D5H): protocol: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection Jul 25 09:34:39.762356667 (21h59m10.952655237s) [seg_pos={0 0}, upload=21166]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:34:39.762441343 (21h59m10.952739924s) [upload=21166]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:39.762447146 (21h59m10.952745714s) [upload=21166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:39.911218699 (21h59m11.101517278s) [upload=21166]: Done uploading segments... Jul 25 09:34:39.911245243 (21h59m11.101543814s) [upload=21166]: Upload finished. err: <nil> Duration: 2.301371108s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.217913825s FAIL (2025-07-25 10:09:56 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.218012359s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.085001786s FAIL (2025-07-25 10:09:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E8772EFA87, HostID: , api error AccessDenied: Access Denied. Duration: 1.085026748s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............XX...............................X.............] duration: 874.072184ms FAIL (2025-07-25 10:09:48 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 874.098967ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.....XX..........................................] duration: 2.349158725s FAIL (2025-07-25 10:09:45 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.34920528s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.36775613s FAIL (2025-07-25 10:09:39 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.367790254s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 909.803543ms FAIL (2025-07-25 10:09:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== g segments... Jul 25 09:26:35.456904650 (21h51m6.64720323s) [upload=21038]: Upload finished. err: <nil> Jul 25 09:27:16.407968928 (21h51m47.598267507s) [upload=21052]: Starting upload Jul 25 09:27:16.408035705 (21h51m47.598334284s) [upload=21052]: Done starting upload Jul 25 09:27:16.408102508 (21h51m47.598401094s) [upload=21052]: Uploading segments... Jul 25 09:27:16.408182298 (21h51m47.598480891s) [upload=21052]: Got next segment. Inline: true Jul 25 09:27:16.408191016 (21h51m47.598489606s) [upload=21052]: Waiting for error group managing segments... Jul 25 09:27:16.408248128 (21h51m47.598546719s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:16.408254857 (21h51m47.598553447s) [upload=21052]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:16.408258740 (21h51m47.598557329s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:16.559578877 (21h51m47.749877447s) [upload=21052]: Done uploading segments... Jul 25 09:27:16.559612194 (21h51m47.749910766s) [upload=21052]: Upload finished. err: <nil> Jul 25 09:28:22.726097098 (21h52m53.916395667s) [upload=21069]: Starting upload Jul 25 09:28:22.726157005 (21h52m53.916455584s) [upload=21069]: Done starting upload Jul 25 09:28:22.726224882 (21h52m53.916523453s) [upload=21069]: Uploading segments... Jul 25 09:28:22.726267961 (21h52m53.916566549s) [upload=21069]: Got next segment. Inline: true Jul 25 09:28:22.726273573 (21h52m53.916572153s) [upload=21069]: Waiting for error group managing segments... Jul 25 09:28:22.726315230 (21h52m53.916613799s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:28:22.726320796 (21h52m53.916619376s) [upload=21069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:28:22.726324173 (21h52m53.916622752s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:28:22.871287048 (21h52m54.06158563s) [upload=21069]: Done uploading segments... Jul 25 09:28:22.871329296 (21h52m54.061627861s) [upload=21069]: Upload finished. err: <nil> Jul 25 09:29:03.680613055 (21h53m34.870911643s) [upload=21081]: Starting upload Jul 25 09:29:03.680698874 (21h53m34.870997445s) [upload=21081]: Done starting upload Jul 25 09:29:03.680794201 (21h53m34.871092783s) [upload=21081]: Uploading segments... Jul 25 09:29:03.680886067 (21h53m34.871184656s) [upload=21081]: Got next segment. Inline: true Jul 25 09:29:03.680906479 (21h53m34.871205059s) [upload=21081]: Waiting for error group managing segments... Jul 25 09:29:03.680962954 (21h53m34.871261535s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:03.680983505 (21h53m34.871282086s) [upload=21081]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:03.680987759 (21h53m34.871286339s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:03.928190102 (21h53m35.118488693s) [upload=21081]: Done uploading segments... Jul 25 09:29:03.928229672 (21h53m35.118528253s) [upload=21081]: Upload finished. err: <nil> Jul 25 09:29:18.766794416 (21h53m49.957092995s) [upload=21086]: Starting upload Jul 25 09:29:18.766860836 (21h53m49.957159459s) [upload=21086]: Done starting upload Jul 25 09:29:18.766928337 (21h53m49.957226924s) [upload=21086]: Uploading segments... Jul 25 09:29:18.767019576 (21h53m49.957318171s) [upload=21086]: Got next segment. Inline: true Jul 25 09:29:18.767028054 (21h53m49.957326646s) [upload=21086]: Waiting for error group managing segments... Jul 25 09:29:18.767082446 (21h53m49.957381032s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:18.767090462 (21h53m49.957389052s) [upload=21086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:18.767095196 (21h53m49.957393783s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:18.966677798 (21h53m50.156976378s) [upload=21086]: Done uploading segments... Jul 25 09:29:18.966694951 (21h53m50.156993524s) [upload=21086]: Upload finished. err: <nil> Jul 25 09:29:49.549110233 (21h54m20.739408822s) [upload=21096]: Starting upload Jul 25 09:29:49.549220943 (21h54m20.739519524s) [upload=21096]: Done starting upload Jul 25 09:29:49.549345183 (21h54m20.739643764s) [upload=21096]: Uploading segments... Jul 25 09:29:49.549394356 (21h54m20.739692958s) [upload=21096]: Got next segment. Inline: true Jul 25 09:29:49.549418287 (21h54m20.73971687s) [upload=21096]: Waiting for error group managing segments... Jul 25 09:29:49.549472215 (21h54m20.739770799s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:49.549491968 (21h54m20.739790549s) [upload=21096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:49.549499684 (21h54m20.739798268s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:49.712461693 (21h54m20.902760273s) [upload=21096]: Done uploading segments... Jul 25 09:29:49.712482595 (21h54m20.902781168s) [upload=21096]: Upload finished. err: <nil> Jul 25 09:30:51.603710658 (21h55m22.794009232s) [upload=21110]: Starting upload Jul 25 09:30:51.603792182 (21h55m22.794090761s) [upload=21110]: Done starting upload Jul 25 09:30:51.603874857 (21h55m22.79417343s) [upload=21110]: Uploading segments... Jul 25 09:30:51.603956850 (21h55m22.794255438s) [upload=21110]: Got next segment. Inline: true Jul 25 09:30:51.603963619 (21h55m22.794262189s) [upload=21110]: Waiting for error group managing segments... Jul 25 09:30:51.604025234 (21h55m22.794323804s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:51.604033463 (21h55m22.794332042s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:51.604038242 (21h55m22.794336822s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:51.761349757 (21h55m22.951648343s) [upload=21110]: Done uploading segments... Jul 25 09:30:51.761389314 (21h55m22.951687896s) [upload=21110]: Upload finished. err: <nil> Jul 25 09:31:17.167122347 (21h55m48.357420933s) [upload=21116]: Starting upload Jul 25 09:31:17.167244983 (21h55m48.357543565s) [upload=21116]: Done starting upload Jul 25 09:31:17.167355538 (21h55m48.357654109s) [upload=21116]: Uploading segments... Jul 25 09:31:17.167422267 (21h55m48.357720857s) [upload=21116]: Got next segment. Inline: true Jul 25 09:31:17.167428785 (21h55m48.357727375s) [upload=21116]: Waiting for error group managing segments... Jul 25 09:31:17.167501643 (21h55m48.357800233s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:17.167510622 (21h55m48.357809211s) [upload=21116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:17.167514153 (21h55m48.357812739s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:17.312004979 (21h55m48.502303564s) [upload=21116]: Done uploading segments... Jul 25 09:31:17.312029374 (21h55m48.502327955s) [upload=21116]: Upload finished. err: <nil> Jul 25 09:32:18.672367175 (21h56m49.862665762s) [upload=21134]: Starting upload Jul 25 09:32:18.672474129 (21h56m49.862772717s) [upload=21134]: Done starting upload Jul 25 09:32:18.672562251 (21h56m49.862860823s) [upload=21134]: Uploading segments... Jul 25 09:32:18.672624817 (21h56m49.862923407s) [upload=21134]: Got next segment. Inline: true Jul 25 09:32:18.672632560 (21h56m49.862931142s) [upload=21134]: Waiting for error group managing segments... Jul 25 09:32:18.672697593 (21h56m49.862996181s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:18.672705554 (21h56m49.86300414s) [upload=21134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:18.672710898 (21h56m49.863009481s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:18.921028738 (21h56m50.111327308s) [upload=21134]: Done uploading segments... Jul 25 09:32:18.921081796 (21h56m50.111380367s) [upload=21134]: Upload finished. err: <nil> Jul 25 09:33:18.245242378 (21h57m49.435540949s) [upload=21145]: Starting upload Jul 25 09:33:18.245313844 (21h57m49.435612415s) [upload=21145]: Done starting upload Jul 25 09:33:18.245389534 (21h57m49.435688106s) [upload=21145]: Uploading segments... Jul 25 09:33:18.245438781 (21h57m49.435737368s) [upload=21145]: Got next segment. Inline: true Jul 25 09:33:18.245444345 (21h57m49.435742915s) [upload=21145]: Waiting for error group managing segments... Jul 25 09:33:18.245508889 (21h57m49.435807471s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:18.245515616 (21h57m49.435814196s) [upload=21145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:18.245520233 (21h57m49.435818808s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:18.386658911 (21h57m49.576957489s) [upload=21145]: Done uploading segments... Jul 25 09:33:18.386751449 (21h57m49.577050032s) [upload=21145]: Upload finished. err: <nil> Jul 25 09:34:10.847242359 (21h58m42.037540934s) [upload=21161]: Starting upload Jul 25 09:34:10.847362462 (21h58m42.037661045s) [upload=21161]: Done starting upload Jul 25 09:34:10.847445757 (21h58m42.037744327s) [upload=21161]: Uploading segments... Jul 25 09:34:10.847489423 (21h58m42.037788008s) [upload=21161]: Got next segment. Inline: true Jul 25 09:34:10.847496016 (21h58m42.037794596s) [upload=21161]: Waiting for error group managing segments... Jul 25 09:34:10.847557787 (21h58m42.037856367s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:10.847563412 (21h58m42.037861992s) [upload=21161]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:34:10.847566753 (21h58m42.037865332s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:10.997618415 (21h58m42.187916989s) [upload=21161]: Done uploading segments... Jul 25 09:34:10.997644278 (21h58m42.187942846s) [upload=21161]: Upload finished. err: <nil> Duration: 909.844551ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 2.5118246s FAIL (2025-07-25 10:09:34 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .BatchRequestItem_SegmentCommit Jul 25 09:34:40.505238896 (21h59m11.695537465s) [upload=21167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:40.640979664 (21h59m11.831278264s) [upload=21167]: Done uploading segments... Jul 25 09:34:40.641009518 (21h59m11.831308091s) [upload=21167]: Upload finished. err: <nil> Jul 25 09:35:20.645390120 (21h59m51.835688706s) [upload=21176]: Starting upload Jul 25 09:35:20.645482113 (21h59m51.835780745s) [upload=21176]: Done starting upload Jul 25 09:35:20.645573260 (21h59m51.835871853s) [upload=21176]: Uploading segments... Jul 25 09:35:20.645792457 (21h59m51.836091041s) [upload=21176]: Got next segment. Inline: false Jul 25 09:35:20.645858035 (21h59m51.836156629s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:20.645865683 (21h59m51.836164267s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:20.754561036 (21h59m51.944859625s) [seg_pos={0 0}, upload=21176]: Begin upload segment... Jul 25 09:35:20.754647616 (21h59m51.944946206s) [seg_pos={0 0}, upload=21176]: Done begin upload segment. Jul 25 09:35:20.754787435 (21h59m51.945086024s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755353972 (21h59m51.94565256s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755471416 (21h59m51.945770007s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756211019 (21h59m51.946509609s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756667323 (21h59m51.946965913s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757099361 (21h59m51.947397948s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757508228 (21h59m51.947806818s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757953278 (21h59m51.948251872s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758367934 (21h59m51.948666523s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758390422 (21h59m51.948689017s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758878977 (21h59m51.949177567s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759244103 (21h59m51.949542684s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759545484 (21h59m51.949844064s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759826303 (21h59m51.950124885s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760136389 (21h59m51.950434969s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760432093 (21h59m51.950730676s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:21.372736599 (21h59m52.563035189s) [upload=21176]: Next returned nil segment Jul 25 09:35:21.372764315 (21h59m52.563062907s) [upload=21176]: Waiting for error group managing segments... Jul 25 09:35:21.511712315 (21h59m52.7020109s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512001132 (21h59m52.702299711s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512309738 (21h59m52.702608319s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.517592678 (21h59m52.707891266s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.519619038 (21h59m52.709917619s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.520794731 (21h59m52.711093299s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.524808060 (21h59m52.715106643s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.527441251 (21h59m52.717739819s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.530273697 (21h59m52.720572278s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.540222474 (21h59m52.730521062s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.541124873 (21h59m52.731423464s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.543682927 (21h59m52.733981514s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549663332 (21h59m52.739961915s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549702269 (21h59m52.740000854s) [seg_pos={0 0}, upload=21176]: Segment reached optimal threshold of 13 pieces. Jul 25 09:35:21.549890135 (21h59m52.740188729s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550054494 (21h59m52.740353094s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550454434 (21h59m52.740753023s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.550506380 (21h59m52.740804969s) [seg_pos={0 0}, upload=21176]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:35:21.550615669 (21h59m52.740914257s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:21.550621910 (21h59m52.740920502s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.698212587 (21h59m52.888511173s) [upload=21176]: Done uploading segments... Jul 25 09:35:21.698237000 (21h59m52.888535586s) [upload=21176]: Upload finished. err: <nil> Jul 25 09:35:50.802823283 (22h0m21.993121863s) [upload=21181]: Starting upload Jul 25 09:35:50.802897121 (22h0m21.993195707s) [upload=21181]: Done starting upload Jul 25 09:35:50.803220257 (22h0m21.993518825s) [upload=21181]: Uploading segments... Jul 25 09:35:50.803284411 (22h0m21.993582999s) [upload=21181]: Got next segment. Inline: false Jul 25 09:35:50.803442141 (22h0m21.993740722s) [upload=21181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:50.803448051 (22h0m21.99374663s) [upload=21181]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:20.533625759 (22h0m51.723924328s) [upload=21181]: Next segment err: context deadline exceeded Jul 25 09:36:20.533795008 (22h0m51.724093581s) [upload=21181]: Done uploading segments... Jul 25 09:36:20.533816345 (22h0m51.724114918s) [upload=21181]: Upload finished. err: context deadline exceeded Duration:* 2.51186155s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 351.32028ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Request-Id=DBm2nRVQhvj Accept-Ranges=bytes Content-Length=279 Server=Storj X-Amz-Request-Id=185574E527952D7A X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:16 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:09:22 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E527952D7A</RequestId><HostId></HostId></Error>" Duration: 351.35292ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.05473334s FAIL (2025-07-25 10:09:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E8772EFA87, HostID: , api error AccessDenied: Access Denied. Duration: 1.054775645s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.632794841s FAIL (2025-07-25 10:09:12 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 21h59m10.08413493s) [seg_pos={0 0}, upload=21166]: Done begin upload segment. Jul 25 09:34:38.893861575 (21h59m10.084160157s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893877564 (21h59m10.084176144s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893970156 (21h59m10.08426874s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894325201 (21h59m10.084623773s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894381124 (21h59m10.084679703s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894891519 (21h59m10.085190109s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895339175 (21h59m10.085637757s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895721362 (21h59m10.086019941s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896080111 (21h59m10.086378691s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896408696 (21h59m10.086707281s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896758554 (21h59m10.087057132s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897033623 (21h59m10.087332202s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897322453 (21h59m10.087621038s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897596601 (21h59m10.087895181s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897880857 (21h59m10.088179437s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898303843 (21h59m10.088602412s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898661671 (21h59m10.08896025s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.899092762 (21h59m10.089391347s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:39.465168779 (21h59m10.655467365s) [upload=21166]: Next returned nil segment Jul 25 09:34:39.465194393 (21h59m10.655492983s) [upload=21166]: Waiting for error group managing segments... Jul 25 09:34:39.598220252 (21h59m10.788518839s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.604836354 (21h59m10.795134975s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.612452239 (21h59m10.80275081s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.619515598 (21h59m10.809814179s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.626845628 (21h59m10.817144209s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.627851431 (21h59m10.818150011s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.629266997 (21h59m10.819565577s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.645882287 (21h59m10.836180867s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.646651873 (21h59m10.836950453s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.647387132 (21h59m10.837685716s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.653519594 (21h59m10.843818177s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.660641895 (21h59m10.850940478s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.711883970 (21h59m10.902182542s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761622344 (21h59m10.951920924s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761651494 (21h59m10.951950073s) [seg_pos={0 0}, upload=21166]: Segment reached optimal threshold of 14 pieces. Jul 25 09:34:39.762107894 (21h59m10.952406474s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:39.762263719 (21h59m10.952562302s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qfp6gac6HdZquSvy2ouWCkMKieFDAN4JUQXg1zHSD3bNCBveV): protocol: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection Jul 25 09:34:39.762288380 (21h59m10.952586956s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TLFJ7HD7rVyLCyaKdy7DAJ1XvdN6s6gg6FuKQ58oJsRV6fhU6): protocol: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection Jul 25 09:34:39.762335180 (21h59m10.952633759s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vyNSWnc7efrHbDqsTmoeuMqg5JrS9dCA1sxEK4J3SfgJV5D5H): protocol: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection Jul 25 09:34:39.762356667 (21h59m10.952655237s) [seg_pos={0 0}, upload=21166]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:34:39.762441343 (21h59m10.952739924s) [upload=21166]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:39.762447146 (21h59m10.952745714s) [upload=21166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:39.911218699 (21h59m11.101517278s) [upload=21166]: Done uploading segments... Jul 25 09:34:39.911245243 (21h59m11.101543814s) [upload=21166]: Upload finished. err: <nil> Duration: 2.632902462s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 889.711539ms FAIL (2025-07-25 10:09:01 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 889.755274ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............XX...............................X............] duration: 831.586488ms FAIL (2025-07-25 10:08:43 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 831.616106ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.....XX.........................................] duration: 2.271672909s FAIL (2025-07-25 10:08:42 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.27170512s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 2.368032107s FAIL (2025-07-25 10:08:40 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .BatchRequestItem_SegmentCommit Jul 25 09:34:40.505238896 (21h59m11.695537465s) [upload=21167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:40.640979664 (21h59m11.831278264s) [upload=21167]: Done uploading segments... Jul 25 09:34:40.641009518 (21h59m11.831308091s) [upload=21167]: Upload finished. err: <nil> Jul 25 09:35:20.645390120 (21h59m51.835688706s) [upload=21176]: Starting upload Jul 25 09:35:20.645482113 (21h59m51.835780745s) [upload=21176]: Done starting upload Jul 25 09:35:20.645573260 (21h59m51.835871853s) [upload=21176]: Uploading segments... Jul 25 09:35:20.645792457 (21h59m51.836091041s) [upload=21176]: Got next segment. Inline: false Jul 25 09:35:20.645858035 (21h59m51.836156629s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:20.645865683 (21h59m51.836164267s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:20.754561036 (21h59m51.944859625s) [seg_pos={0 0}, upload=21176]: Begin upload segment... Jul 25 09:35:20.754647616 (21h59m51.944946206s) [seg_pos={0 0}, upload=21176]: Done begin upload segment. Jul 25 09:35:20.754787435 (21h59m51.945086024s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755353972 (21h59m51.94565256s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755471416 (21h59m51.945770007s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756211019 (21h59m51.946509609s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756667323 (21h59m51.946965913s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757099361 (21h59m51.947397948s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757508228 (21h59m51.947806818s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757953278 (21h59m51.948251872s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758367934 (21h59m51.948666523s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758390422 (21h59m51.948689017s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758878977 (21h59m51.949177567s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759244103 (21h59m51.949542684s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759545484 (21h59m51.949844064s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759826303 (21h59m51.950124885s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760136389 (21h59m51.950434969s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760432093 (21h59m51.950730676s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:21.372736599 (21h59m52.563035189s) [upload=21176]: Next returned nil segment Jul 25 09:35:21.372764315 (21h59m52.563062907s) [upload=21176]: Waiting for error group managing segments... Jul 25 09:35:21.511712315 (21h59m52.7020109s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512001132 (21h59m52.702299711s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512309738 (21h59m52.702608319s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.517592678 (21h59m52.707891266s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.519619038 (21h59m52.709917619s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.520794731 (21h59m52.711093299s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.524808060 (21h59m52.715106643s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.527441251 (21h59m52.717739819s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.530273697 (21h59m52.720572278s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.540222474 (21h59m52.730521062s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.541124873 (21h59m52.731423464s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.543682927 (21h59m52.733981514s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549663332 (21h59m52.739961915s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549702269 (21h59m52.740000854s) [seg_pos={0 0}, upload=21176]: Segment reached optimal threshold of 13 pieces. Jul 25 09:35:21.549890135 (21h59m52.740188729s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550054494 (21h59m52.740353094s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550454434 (21h59m52.740753023s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.550506380 (21h59m52.740804969s) [seg_pos={0 0}, upload=21176]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:35:21.550615669 (21h59m52.740914257s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:21.550621910 (21h59m52.740920502s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.698212587 (21h59m52.888511173s) [upload=21176]: Done uploading segments... Jul 25 09:35:21.698237000 (21h59m52.888535586s) [upload=21176]: Upload finished. err: <nil> Jul 25 09:35:50.802823283 (22h0m21.993121863s) [upload=21181]: Starting upload Jul 25 09:35:50.802897121 (22h0m21.993195707s) [upload=21181]: Done starting upload Jul 25 09:35:50.803220257 (22h0m21.993518825s) [upload=21181]: Uploading segments... Jul 25 09:35:50.803284411 (22h0m21.993582999s) [upload=21181]: Got next segment. Inline: false Jul 25 09:35:50.803442141 (22h0m21.993740722s) [upload=21181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:50.803448051 (22h0m21.99374663s) [upload=21181]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:20.533625759 (22h0m51.723924328s) [upload=21181]: Next segment err: context deadline exceeded Jul 25 09:36:20.533795008 (22h0m51.724093581s) [upload=21181]: Done uploading segments... Jul 25 09:36:20.533816345 (22h0m51.724114918s) [upload=21181]: Upload finished. err: context deadline exceeded Duration:* 2.368149643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 953.93062ms FAIL (2025-07-25 10:08:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== g segments... Jul 25 09:26:35.456904650 (21h51m6.64720323s) [upload=21038]: Upload finished. err: <nil> Jul 25 09:27:16.407968928 (21h51m47.598267507s) [upload=21052]: Starting upload Jul 25 09:27:16.408035705 (21h51m47.598334284s) [upload=21052]: Done starting upload Jul 25 09:27:16.408102508 (21h51m47.598401094s) [upload=21052]: Uploading segments... Jul 25 09:27:16.408182298 (21h51m47.598480891s) [upload=21052]: Got next segment. Inline: true Jul 25 09:27:16.408191016 (21h51m47.598489606s) [upload=21052]: Waiting for error group managing segments... Jul 25 09:27:16.408248128 (21h51m47.598546719s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:16.408254857 (21h51m47.598553447s) [upload=21052]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:16.408258740 (21h51m47.598557329s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:16.559578877 (21h51m47.749877447s) [upload=21052]: Done uploading segments... Jul 25 09:27:16.559612194 (21h51m47.749910766s) [upload=21052]: Upload finished. err: <nil> Jul 25 09:28:22.726097098 (21h52m53.916395667s) [upload=21069]: Starting upload Jul 25 09:28:22.726157005 (21h52m53.916455584s) [upload=21069]: Done starting upload Jul 25 09:28:22.726224882 (21h52m53.916523453s) [upload=21069]: Uploading segments... Jul 25 09:28:22.726267961 (21h52m53.916566549s) [upload=21069]: Got next segment. Inline: true Jul 25 09:28:22.726273573 (21h52m53.916572153s) [upload=21069]: Waiting for error group managing segments... Jul 25 09:28:22.726315230 (21h52m53.916613799s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:28:22.726320796 (21h52m53.916619376s) [upload=21069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:28:22.726324173 (21h52m53.916622752s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:28:22.871287048 (21h52m54.06158563s) [upload=21069]: Done uploading segments... Jul 25 09:28:22.871329296 (21h52m54.061627861s) [upload=21069]: Upload finished. err: <nil> Jul 25 09:29:03.680613055 (21h53m34.870911643s) [upload=21081]: Starting upload Jul 25 09:29:03.680698874 (21h53m34.870997445s) [upload=21081]: Done starting upload Jul 25 09:29:03.680794201 (21h53m34.871092783s) [upload=21081]: Uploading segments... Jul 25 09:29:03.680886067 (21h53m34.871184656s) [upload=21081]: Got next segment. Inline: true Jul 25 09:29:03.680906479 (21h53m34.871205059s) [upload=21081]: Waiting for error group managing segments... Jul 25 09:29:03.680962954 (21h53m34.871261535s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:03.680983505 (21h53m34.871282086s) [upload=21081]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:03.680987759 (21h53m34.871286339s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:03.928190102 (21h53m35.118488693s) [upload=21081]: Done uploading segments... Jul 25 09:29:03.928229672 (21h53m35.118528253s) [upload=21081]: Upload finished. err: <nil> Jul 25 09:29:18.766794416 (21h53m49.957092995s) [upload=21086]: Starting upload Jul 25 09:29:18.766860836 (21h53m49.957159459s) [upload=21086]: Done starting upload Jul 25 09:29:18.766928337 (21h53m49.957226924s) [upload=21086]: Uploading segments... Jul 25 09:29:18.767019576 (21h53m49.957318171s) [upload=21086]: Got next segment. Inline: true Jul 25 09:29:18.767028054 (21h53m49.957326646s) [upload=21086]: Waiting for error group managing segments... Jul 25 09:29:18.767082446 (21h53m49.957381032s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:18.767090462 (21h53m49.957389052s) [upload=21086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:18.767095196 (21h53m49.957393783s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:18.966677798 (21h53m50.156976378s) [upload=21086]: Done uploading segments... Jul 25 09:29:18.966694951 (21h53m50.156993524s) [upload=21086]: Upload finished. err: <nil> Jul 25 09:29:49.549110233 (21h54m20.739408822s) [upload=21096]: Starting upload Jul 25 09:29:49.549220943 (21h54m20.739519524s) [upload=21096]: Done starting upload Jul 25 09:29:49.549345183 (21h54m20.739643764s) [upload=21096]: Uploading segments... Jul 25 09:29:49.549394356 (21h54m20.739692958s) [upload=21096]: Got next segment. Inline: true Jul 25 09:29:49.549418287 (21h54m20.73971687s) [upload=21096]: Waiting for error group managing segments... Jul 25 09:29:49.549472215 (21h54m20.739770799s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:49.549491968 (21h54m20.739790549s) [upload=21096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:49.549499684 (21h54m20.739798268s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:49.712461693 (21h54m20.902760273s) [upload=21096]: Done uploading segments... Jul 25 09:29:49.712482595 (21h54m20.902781168s) [upload=21096]: Upload finished. err: <nil> Jul 25 09:30:51.603710658 (21h55m22.794009232s) [upload=21110]: Starting upload Jul 25 09:30:51.603792182 (21h55m22.794090761s) [upload=21110]: Done starting upload Jul 25 09:30:51.603874857 (21h55m22.79417343s) [upload=21110]: Uploading segments... Jul 25 09:30:51.603956850 (21h55m22.794255438s) [upload=21110]: Got next segment. Inline: true Jul 25 09:30:51.603963619 (21h55m22.794262189s) [upload=21110]: Waiting for error group managing segments... Jul 25 09:30:51.604025234 (21h55m22.794323804s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:51.604033463 (21h55m22.794332042s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:51.604038242 (21h55m22.794336822s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:51.761349757 (21h55m22.951648343s) [upload=21110]: Done uploading segments... Jul 25 09:30:51.761389314 (21h55m22.951687896s) [upload=21110]: Upload finished. err: <nil> Jul 25 09:31:17.167122347 (21h55m48.357420933s) [upload=21116]: Starting upload Jul 25 09:31:17.167244983 (21h55m48.357543565s) [upload=21116]: Done starting upload Jul 25 09:31:17.167355538 (21h55m48.357654109s) [upload=21116]: Uploading segments... Jul 25 09:31:17.167422267 (21h55m48.357720857s) [upload=21116]: Got next segment. Inline: true Jul 25 09:31:17.167428785 (21h55m48.357727375s) [upload=21116]: Waiting for error group managing segments... Jul 25 09:31:17.167501643 (21h55m48.357800233s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:17.167510622 (21h55m48.357809211s) [upload=21116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:17.167514153 (21h55m48.357812739s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:17.312004979 (21h55m48.502303564s) [upload=21116]: Done uploading segments... Jul 25 09:31:17.312029374 (21h55m48.502327955s) [upload=21116]: Upload finished. err: <nil> Jul 25 09:32:18.672367175 (21h56m49.862665762s) [upload=21134]: Starting upload Jul 25 09:32:18.672474129 (21h56m49.862772717s) [upload=21134]: Done starting upload Jul 25 09:32:18.672562251 (21h56m49.862860823s) [upload=21134]: Uploading segments... Jul 25 09:32:18.672624817 (21h56m49.862923407s) [upload=21134]: Got next segment. Inline: true Jul 25 09:32:18.672632560 (21h56m49.862931142s) [upload=21134]: Waiting for error group managing segments... Jul 25 09:32:18.672697593 (21h56m49.862996181s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:18.672705554 (21h56m49.86300414s) [upload=21134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:18.672710898 (21h56m49.863009481s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:18.921028738 (21h56m50.111327308s) [upload=21134]: Done uploading segments... Jul 25 09:32:18.921081796 (21h56m50.111380367s) [upload=21134]: Upload finished. err: <nil> Jul 25 09:33:18.245242378 (21h57m49.435540949s) [upload=21145]: Starting upload Jul 25 09:33:18.245313844 (21h57m49.435612415s) [upload=21145]: Done starting upload Jul 25 09:33:18.245389534 (21h57m49.435688106s) [upload=21145]: Uploading segments... Jul 25 09:33:18.245438781 (21h57m49.435737368s) [upload=21145]: Got next segment. Inline: true Jul 25 09:33:18.245444345 (21h57m49.435742915s) [upload=21145]: Waiting for error group managing segments... Jul 25 09:33:18.245508889 (21h57m49.435807471s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:18.245515616 (21h57m49.435814196s) [upload=21145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:18.245520233 (21h57m49.435818808s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:18.386658911 (21h57m49.576957489s) [upload=21145]: Done uploading segments... Jul 25 09:33:18.386751449 (21h57m49.577050032s) [upload=21145]: Upload finished. err: <nil> Jul 25 09:34:10.847242359 (21h58m42.037540934s) [upload=21161]: Starting upload Jul 25 09:34:10.847362462 (21h58m42.037661045s) [upload=21161]: Done starting upload Jul 25 09:34:10.847445757 (21h58m42.037744327s) [upload=21161]: Uploading segments... Jul 25 09:34:10.847489423 (21h58m42.037788008s) [upload=21161]: Got next segment. Inline: true Jul 25 09:34:10.847496016 (21h58m42.037794596s) [upload=21161]: Waiting for error group managing segments... Jul 25 09:34:10.847557787 (21h58m42.037856367s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:10.847563412 (21h58m42.037861992s) [upload=21161]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:34:10.847566753 (21h58m42.037865332s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:10.997618415 (21h58m42.187916989s) [upload=21161]: Done uploading segments... Jul 25 09:34:10.997644278 (21h58m42.187942846s) [upload=21161]: Upload finished. err: <nil> Duration: 953.964099ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.374964033s FAIL (2025-07-25 10:08:29 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.374993432s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.....XX........................................] duration: 2.300022427s FAIL (2025-07-25 10:08:27 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.30005321s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............XX...............................X...........] duration: 1.038787297s FAIL (2025-07-25 10:08:24 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 1.038836405s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 3.32378406s FAIL (2025-07-25 10:08:15 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 21h59m10.08413493s) [seg_pos={0 0}, upload=21166]: Done begin upload segment. Jul 25 09:34:38.893861575 (21h59m10.084160157s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893877564 (21h59m10.084176144s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893970156 (21h59m10.08426874s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894325201 (21h59m10.084623773s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894381124 (21h59m10.084679703s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894891519 (21h59m10.085190109s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895339175 (21h59m10.085637757s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895721362 (21h59m10.086019941s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896080111 (21h59m10.086378691s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896408696 (21h59m10.086707281s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896758554 (21h59m10.087057132s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897033623 (21h59m10.087332202s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897322453 (21h59m10.087621038s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897596601 (21h59m10.087895181s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897880857 (21h59m10.088179437s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898303843 (21h59m10.088602412s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898661671 (21h59m10.08896025s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.899092762 (21h59m10.089391347s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:39.465168779 (21h59m10.655467365s) [upload=21166]: Next returned nil segment Jul 25 09:34:39.465194393 (21h59m10.655492983s) [upload=21166]: Waiting for error group managing segments... Jul 25 09:34:39.598220252 (21h59m10.788518839s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.604836354 (21h59m10.795134975s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.612452239 (21h59m10.80275081s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.619515598 (21h59m10.809814179s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.626845628 (21h59m10.817144209s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.627851431 (21h59m10.818150011s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.629266997 (21h59m10.819565577s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.645882287 (21h59m10.836180867s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.646651873 (21h59m10.836950453s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.647387132 (21h59m10.837685716s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.653519594 (21h59m10.843818177s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.660641895 (21h59m10.850940478s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.711883970 (21h59m10.902182542s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761622344 (21h59m10.951920924s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761651494 (21h59m10.951950073s) [seg_pos={0 0}, upload=21166]: Segment reached optimal threshold of 14 pieces. Jul 25 09:34:39.762107894 (21h59m10.952406474s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:39.762263719 (21h59m10.952562302s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qfp6gac6HdZquSvy2ouWCkMKieFDAN4JUQXg1zHSD3bNCBveV): protocol: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection Jul 25 09:34:39.762288380 (21h59m10.952586956s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TLFJ7HD7rVyLCyaKdy7DAJ1XvdN6s6gg6FuKQ58oJsRV6fhU6): protocol: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection Jul 25 09:34:39.762335180 (21h59m10.952633759s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vyNSWnc7efrHbDqsTmoeuMqg5JrS9dCA1sxEK4J3SfgJV5D5H): protocol: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection Jul 25 09:34:39.762356667 (21h59m10.952655237s) [seg_pos={0 0}, upload=21166]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:34:39.762441343 (21h59m10.952739924s) [upload=21166]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:39.762447146 (21h59m10.952745714s) [upload=21166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:39.911218699 (21h59m11.101517278s) [upload=21166]: Done uploading segments... Jul 25 09:34:39.911245243 (21h59m11.101543814s) [upload=21166]: Upload finished. err: <nil> Duration: 3.323892755s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 341.902549ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Request-Id=DBm2nRVQhvj Accept-Ranges=bytes Content-Length=279 Server=Storj X-Amz-Request-Id=185574E527952D7A X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:16 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:08:15 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E527952D7A</RequestId><HostId></HostId></Error>" Duration: 341.950549ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.196151958s FAIL (2025-07-25 10:08:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E8772EFA87, HostID: , api error AccessDenied: Access Denied. Duration: 1.196221375s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 927.935519ms FAIL (2025-07-25 10:08:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== g segments... Jul 25 09:26:35.456904650 (21h51m6.64720323s) [upload=21038]: Upload finished. err: <nil> Jul 25 09:27:16.407968928 (21h51m47.598267507s) [upload=21052]: Starting upload Jul 25 09:27:16.408035705 (21h51m47.598334284s) [upload=21052]: Done starting upload Jul 25 09:27:16.408102508 (21h51m47.598401094s) [upload=21052]: Uploading segments... Jul 25 09:27:16.408182298 (21h51m47.598480891s) [upload=21052]: Got next segment. Inline: true Jul 25 09:27:16.408191016 (21h51m47.598489606s) [upload=21052]: Waiting for error group managing segments... Jul 25 09:27:16.408248128 (21h51m47.598546719s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:16.408254857 (21h51m47.598553447s) [upload=21052]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:16.408258740 (21h51m47.598557329s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:16.559578877 (21h51m47.749877447s) [upload=21052]: Done uploading segments... Jul 25 09:27:16.559612194 (21h51m47.749910766s) [upload=21052]: Upload finished. err: <nil> Jul 25 09:28:22.726097098 (21h52m53.916395667s) [upload=21069]: Starting upload Jul 25 09:28:22.726157005 (21h52m53.916455584s) [upload=21069]: Done starting upload Jul 25 09:28:22.726224882 (21h52m53.916523453s) [upload=21069]: Uploading segments... Jul 25 09:28:22.726267961 (21h52m53.916566549s) [upload=21069]: Got next segment. Inline: true Jul 25 09:28:22.726273573 (21h52m53.916572153s) [upload=21069]: Waiting for error group managing segments... Jul 25 09:28:22.726315230 (21h52m53.916613799s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:28:22.726320796 (21h52m53.916619376s) [upload=21069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:28:22.726324173 (21h52m53.916622752s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:28:22.871287048 (21h52m54.06158563s) [upload=21069]: Done uploading segments... Jul 25 09:28:22.871329296 (21h52m54.061627861s) [upload=21069]: Upload finished. err: <nil> Jul 25 09:29:03.680613055 (21h53m34.870911643s) [upload=21081]: Starting upload Jul 25 09:29:03.680698874 (21h53m34.870997445s) [upload=21081]: Done starting upload Jul 25 09:29:03.680794201 (21h53m34.871092783s) [upload=21081]: Uploading segments... Jul 25 09:29:03.680886067 (21h53m34.871184656s) [upload=21081]: Got next segment. Inline: true Jul 25 09:29:03.680906479 (21h53m34.871205059s) [upload=21081]: Waiting for error group managing segments... Jul 25 09:29:03.680962954 (21h53m34.871261535s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:03.680983505 (21h53m34.871282086s) [upload=21081]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:03.680987759 (21h53m34.871286339s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:03.928190102 (21h53m35.118488693s) [upload=21081]: Done uploading segments... Jul 25 09:29:03.928229672 (21h53m35.118528253s) [upload=21081]: Upload finished. err: <nil> Jul 25 09:29:18.766794416 (21h53m49.957092995s) [upload=21086]: Starting upload Jul 25 09:29:18.766860836 (21h53m49.957159459s) [upload=21086]: Done starting upload Jul 25 09:29:18.766928337 (21h53m49.957226924s) [upload=21086]: Uploading segments... Jul 25 09:29:18.767019576 (21h53m49.957318171s) [upload=21086]: Got next segment. Inline: true Jul 25 09:29:18.767028054 (21h53m49.957326646s) [upload=21086]: Waiting for error group managing segments... Jul 25 09:29:18.767082446 (21h53m49.957381032s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:18.767090462 (21h53m49.957389052s) [upload=21086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:18.767095196 (21h53m49.957393783s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:18.966677798 (21h53m50.156976378s) [upload=21086]: Done uploading segments... Jul 25 09:29:18.966694951 (21h53m50.156993524s) [upload=21086]: Upload finished. err: <nil> Jul 25 09:29:49.549110233 (21h54m20.739408822s) [upload=21096]: Starting upload Jul 25 09:29:49.549220943 (21h54m20.739519524s) [upload=21096]: Done starting upload Jul 25 09:29:49.549345183 (21h54m20.739643764s) [upload=21096]: Uploading segments... Jul 25 09:29:49.549394356 (21h54m20.739692958s) [upload=21096]: Got next segment. Inline: true Jul 25 09:29:49.549418287 (21h54m20.73971687s) [upload=21096]: Waiting for error group managing segments... Jul 25 09:29:49.549472215 (21h54m20.739770799s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:49.549491968 (21h54m20.739790549s) [upload=21096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:49.549499684 (21h54m20.739798268s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:49.712461693 (21h54m20.902760273s) [upload=21096]: Done uploading segments... Jul 25 09:29:49.712482595 (21h54m20.902781168s) [upload=21096]: Upload finished. err: <nil> Jul 25 09:30:51.603710658 (21h55m22.794009232s) [upload=21110]: Starting upload Jul 25 09:30:51.603792182 (21h55m22.794090761s) [upload=21110]: Done starting upload Jul 25 09:30:51.603874857 (21h55m22.79417343s) [upload=21110]: Uploading segments... Jul 25 09:30:51.603956850 (21h55m22.794255438s) [upload=21110]: Got next segment. Inline: true Jul 25 09:30:51.603963619 (21h55m22.794262189s) [upload=21110]: Waiting for error group managing segments... Jul 25 09:30:51.604025234 (21h55m22.794323804s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:51.604033463 (21h55m22.794332042s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:51.604038242 (21h55m22.794336822s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:51.761349757 (21h55m22.951648343s) [upload=21110]: Done uploading segments... Jul 25 09:30:51.761389314 (21h55m22.951687896s) [upload=21110]: Upload finished. err: <nil> Jul 25 09:31:17.167122347 (21h55m48.357420933s) [upload=21116]: Starting upload Jul 25 09:31:17.167244983 (21h55m48.357543565s) [upload=21116]: Done starting upload Jul 25 09:31:17.167355538 (21h55m48.357654109s) [upload=21116]: Uploading segments... Jul 25 09:31:17.167422267 (21h55m48.357720857s) [upload=21116]: Got next segment. Inline: true Jul 25 09:31:17.167428785 (21h55m48.357727375s) [upload=21116]: Waiting for error group managing segments... Jul 25 09:31:17.167501643 (21h55m48.357800233s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:17.167510622 (21h55m48.357809211s) [upload=21116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:17.167514153 (21h55m48.357812739s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:17.312004979 (21h55m48.502303564s) [upload=21116]: Done uploading segments... Jul 25 09:31:17.312029374 (21h55m48.502327955s) [upload=21116]: Upload finished. err: <nil> Jul 25 09:32:18.672367175 (21h56m49.862665762s) [upload=21134]: Starting upload Jul 25 09:32:18.672474129 (21h56m49.862772717s) [upload=21134]: Done starting upload Jul 25 09:32:18.672562251 (21h56m49.862860823s) [upload=21134]: Uploading segments... Jul 25 09:32:18.672624817 (21h56m49.862923407s) [upload=21134]: Got next segment. Inline: true Jul 25 09:32:18.672632560 (21h56m49.862931142s) [upload=21134]: Waiting for error group managing segments... Jul 25 09:32:18.672697593 (21h56m49.862996181s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:18.672705554 (21h56m49.86300414s) [upload=21134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:18.672710898 (21h56m49.863009481s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:18.921028738 (21h56m50.111327308s) [upload=21134]: Done uploading segments... Jul 25 09:32:18.921081796 (21h56m50.111380367s) [upload=21134]: Upload finished. err: <nil> Jul 25 09:33:18.245242378 (21h57m49.435540949s) [upload=21145]: Starting upload Jul 25 09:33:18.245313844 (21h57m49.435612415s) [upload=21145]: Done starting upload Jul 25 09:33:18.245389534 (21h57m49.435688106s) [upload=21145]: Uploading segments... Jul 25 09:33:18.245438781 (21h57m49.435737368s) [upload=21145]: Got next segment. Inline: true Jul 25 09:33:18.245444345 (21h57m49.435742915s) [upload=21145]: Waiting for error group managing segments... Jul 25 09:33:18.245508889 (21h57m49.435807471s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:18.245515616 (21h57m49.435814196s) [upload=21145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:18.245520233 (21h57m49.435818808s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:18.386658911 (21h57m49.576957489s) [upload=21145]: Done uploading segments... Jul 25 09:33:18.386751449 (21h57m49.577050032s) [upload=21145]: Upload finished. err: <nil> Jul 25 09:34:10.847242359 (21h58m42.037540934s) [upload=21161]: Starting upload Jul 25 09:34:10.847362462 (21h58m42.037661045s) [upload=21161]: Done starting upload Jul 25 09:34:10.847445757 (21h58m42.037744327s) [upload=21161]: Uploading segments... Jul 25 09:34:10.847489423 (21h58m42.037788008s) [upload=21161]: Got next segment. Inline: true Jul 25 09:34:10.847496016 (21h58m42.037794596s) [upload=21161]: Waiting for error group managing segments... Jul 25 09:34:10.847557787 (21h58m42.037856367s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:10.847563412 (21h58m42.037861992s) [upload=21161]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:34:10.847566753 (21h58m42.037865332s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:10.997618415 (21h58m42.187916989s) [upload=21161]: Done uploading segments... Jul 25 09:34:10.997644278 (21h58m42.187942846s) [upload=21161]: Upload finished. err: <nil> Duration: 927.965092ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 2.595588608s FAIL (2025-07-25 10:08:02 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .BatchRequestItem_SegmentCommit Jul 25 09:34:40.505238896 (21h59m11.695537465s) [upload=21167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:40.640979664 (21h59m11.831278264s) [upload=21167]: Done uploading segments... Jul 25 09:34:40.641009518 (21h59m11.831308091s) [upload=21167]: Upload finished. err: <nil> Jul 25 09:35:20.645390120 (21h59m51.835688706s) [upload=21176]: Starting upload Jul 25 09:35:20.645482113 (21h59m51.835780745s) [upload=21176]: Done starting upload Jul 25 09:35:20.645573260 (21h59m51.835871853s) [upload=21176]: Uploading segments... Jul 25 09:35:20.645792457 (21h59m51.836091041s) [upload=21176]: Got next segment. Inline: false Jul 25 09:35:20.645858035 (21h59m51.836156629s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:20.645865683 (21h59m51.836164267s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:20.754561036 (21h59m51.944859625s) [seg_pos={0 0}, upload=21176]: Begin upload segment... Jul 25 09:35:20.754647616 (21h59m51.944946206s) [seg_pos={0 0}, upload=21176]: Done begin upload segment. Jul 25 09:35:20.754787435 (21h59m51.945086024s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755353972 (21h59m51.94565256s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755471416 (21h59m51.945770007s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756211019 (21h59m51.946509609s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756667323 (21h59m51.946965913s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757099361 (21h59m51.947397948s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757508228 (21h59m51.947806818s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757953278 (21h59m51.948251872s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758367934 (21h59m51.948666523s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758390422 (21h59m51.948689017s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758878977 (21h59m51.949177567s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759244103 (21h59m51.949542684s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759545484 (21h59m51.949844064s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759826303 (21h59m51.950124885s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760136389 (21h59m51.950434969s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760432093 (21h59m51.950730676s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:21.372736599 (21h59m52.563035189s) [upload=21176]: Next returned nil segment Jul 25 09:35:21.372764315 (21h59m52.563062907s) [upload=21176]: Waiting for error group managing segments... Jul 25 09:35:21.511712315 (21h59m52.7020109s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512001132 (21h59m52.702299711s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512309738 (21h59m52.702608319s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.517592678 (21h59m52.707891266s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.519619038 (21h59m52.709917619s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.520794731 (21h59m52.711093299s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.524808060 (21h59m52.715106643s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.527441251 (21h59m52.717739819s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.530273697 (21h59m52.720572278s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.540222474 (21h59m52.730521062s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.541124873 (21h59m52.731423464s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.543682927 (21h59m52.733981514s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549663332 (21h59m52.739961915s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549702269 (21h59m52.740000854s) [seg_pos={0 0}, upload=21176]: Segment reached optimal threshold of 13 pieces. Jul 25 09:35:21.549890135 (21h59m52.740188729s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550054494 (21h59m52.740353094s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550454434 (21h59m52.740753023s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.550506380 (21h59m52.740804969s) [seg_pos={0 0}, upload=21176]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:35:21.550615669 (21h59m52.740914257s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:21.550621910 (21h59m52.740920502s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.698212587 (21h59m52.888511173s) [upload=21176]: Done uploading segments... Jul 25 09:35:21.698237000 (21h59m52.888535586s) [upload=21176]: Upload finished. err: <nil> Jul 25 09:35:50.802823283 (22h0m21.993121863s) [upload=21181]: Starting upload Jul 25 09:35:50.802897121 (22h0m21.993195707s) [upload=21181]: Done starting upload Jul 25 09:35:50.803220257 (22h0m21.993518825s) [upload=21181]: Uploading segments... Jul 25 09:35:50.803284411 (22h0m21.993582999s) [upload=21181]: Got next segment. Inline: false Jul 25 09:35:50.803442141 (22h0m21.993740722s) [upload=21181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:50.803448051 (22h0m21.99374663s) [upload=21181]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:20.533625759 (22h0m51.723924328s) [upload=21181]: Next segment err: context deadline exceeded Jul 25 09:36:20.533795008 (22h0m51.724093581s) [upload=21181]: Done uploading segments... Jul 25 09:36:20.533816345 (22h0m51.724114918s) [upload=21181]: Upload finished. err: context deadline exceeded Duration:* 2.595617767s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [................XX...............................X..........] duration: 915.849608ms FAIL (2025-07-25 10:08:00 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 915.95571ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 1.044473168s FAIL (2025-07-25 10:07:55 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 1.044508956s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.449436341s FAIL (2025-07-25 10:07:49 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 21h59m10.08413493s) [seg_pos={0 0}, upload=21166]: Done begin upload segment. Jul 25 09:34:38.893861575 (21h59m10.084160157s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893877564 (21h59m10.084176144s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893970156 (21h59m10.08426874s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894325201 (21h59m10.084623773s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894381124 (21h59m10.084679703s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894891519 (21h59m10.085190109s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895339175 (21h59m10.085637757s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895721362 (21h59m10.086019941s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896080111 (21h59m10.086378691s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896408696 (21h59m10.086707281s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896758554 (21h59m10.087057132s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897033623 (21h59m10.087332202s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897322453 (21h59m10.087621038s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897596601 (21h59m10.087895181s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897880857 (21h59m10.088179437s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898303843 (21h59m10.088602412s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898661671 (21h59m10.08896025s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.899092762 (21h59m10.089391347s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:39.465168779 (21h59m10.655467365s) [upload=21166]: Next returned nil segment Jul 25 09:34:39.465194393 (21h59m10.655492983s) [upload=21166]: Waiting for error group managing segments... Jul 25 09:34:39.598220252 (21h59m10.788518839s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.604836354 (21h59m10.795134975s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.612452239 (21h59m10.80275081s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.619515598 (21h59m10.809814179s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.626845628 (21h59m10.817144209s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.627851431 (21h59m10.818150011s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.629266997 (21h59m10.819565577s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.645882287 (21h59m10.836180867s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.646651873 (21h59m10.836950453s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.647387132 (21h59m10.837685716s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.653519594 (21h59m10.843818177s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.660641895 (21h59m10.850940478s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.711883970 (21h59m10.902182542s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761622344 (21h59m10.951920924s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761651494 (21h59m10.951950073s) [seg_pos={0 0}, upload=21166]: Segment reached optimal threshold of 14 pieces. Jul 25 09:34:39.762107894 (21h59m10.952406474s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:39.762263719 (21h59m10.952562302s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qfp6gac6HdZquSvy2ouWCkMKieFDAN4JUQXg1zHSD3bNCBveV): protocol: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection Jul 25 09:34:39.762288380 (21h59m10.952586956s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TLFJ7HD7rVyLCyaKdy7DAJ1XvdN6s6gg6FuKQ58oJsRV6fhU6): protocol: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection Jul 25 09:34:39.762335180 (21h59m10.952633759s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vyNSWnc7efrHbDqsTmoeuMqg5JrS9dCA1sxEK4J3SfgJV5D5H): protocol: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection Jul 25 09:34:39.762356667 (21h59m10.952655237s) [seg_pos={0 0}, upload=21166]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:34:39.762441343 (21h59m10.952739924s) [upload=21166]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:39.762447146 (21h59m10.952745714s) [upload=21166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:39.911218699 (21h59m11.101517278s) [upload=21166]: Done uploading segments... Jul 25 09:34:39.911245243 (21h59m11.101543814s) [upload=21166]: Upload finished. err: <nil> Duration: 2.449469874s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 315.066155ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Request-Id=DBm2nRVQhvj Accept-Ranges=bytes Content-Length=279 Server=Storj X-Amz-Request-Id=185574E527952D7A X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:16 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:07:31 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E527952D7A</RequestId><HostId></HostId></Error>" Duration: 315.09787ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.156015649s FAIL (2025-07-25 10:07:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E8772EFA87, HostID: , api error AccessDenied: Access Denied. Duration: 1.156035651s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.467800849s FAIL (2025-07-25 10:07:25 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.46784733s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 887.864775ms FAIL (2025-07-25 10:07:23 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 887.894773ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X.....XX.......................................] duration: 2.283701806s FAIL (2025-07-25 10:07:16 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.283742957s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 2.400620361s FAIL (2025-07-25 10:07:08 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .BatchRequestItem_SegmentCommit Jul 25 09:34:40.505238896 (21h59m11.695537465s) [upload=21167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:40.640979664 (21h59m11.831278264s) [upload=21167]: Done uploading segments... Jul 25 09:34:40.641009518 (21h59m11.831308091s) [upload=21167]: Upload finished. err: <nil> Jul 25 09:35:20.645390120 (21h59m51.835688706s) [upload=21176]: Starting upload Jul 25 09:35:20.645482113 (21h59m51.835780745s) [upload=21176]: Done starting upload Jul 25 09:35:20.645573260 (21h59m51.835871853s) [upload=21176]: Uploading segments... Jul 25 09:35:20.645792457 (21h59m51.836091041s) [upload=21176]: Got next segment. Inline: false Jul 25 09:35:20.645858035 (21h59m51.836156629s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:20.645865683 (21h59m51.836164267s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:20.754561036 (21h59m51.944859625s) [seg_pos={0 0}, upload=21176]: Begin upload segment... Jul 25 09:35:20.754647616 (21h59m51.944946206s) [seg_pos={0 0}, upload=21176]: Done begin upload segment. Jul 25 09:35:20.754787435 (21h59m51.945086024s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755353972 (21h59m51.94565256s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755471416 (21h59m51.945770007s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756211019 (21h59m51.946509609s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756667323 (21h59m51.946965913s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757099361 (21h59m51.947397948s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757508228 (21h59m51.947806818s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757953278 (21h59m51.948251872s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758367934 (21h59m51.948666523s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758390422 (21h59m51.948689017s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758878977 (21h59m51.949177567s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759244103 (21h59m51.949542684s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759545484 (21h59m51.949844064s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759826303 (21h59m51.950124885s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760136389 (21h59m51.950434969s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760432093 (21h59m51.950730676s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:21.372736599 (21h59m52.563035189s) [upload=21176]: Next returned nil segment Jul 25 09:35:21.372764315 (21h59m52.563062907s) [upload=21176]: Waiting for error group managing segments... Jul 25 09:35:21.511712315 (21h59m52.7020109s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512001132 (21h59m52.702299711s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512309738 (21h59m52.702608319s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.517592678 (21h59m52.707891266s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.519619038 (21h59m52.709917619s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.520794731 (21h59m52.711093299s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.524808060 (21h59m52.715106643s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.527441251 (21h59m52.717739819s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.530273697 (21h59m52.720572278s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.540222474 (21h59m52.730521062s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.541124873 (21h59m52.731423464s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.543682927 (21h59m52.733981514s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549663332 (21h59m52.739961915s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549702269 (21h59m52.740000854s) [seg_pos={0 0}, upload=21176]: Segment reached optimal threshold of 13 pieces. Jul 25 09:35:21.549890135 (21h59m52.740188729s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550054494 (21h59m52.740353094s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550454434 (21h59m52.740753023s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.550506380 (21h59m52.740804969s) [seg_pos={0 0}, upload=21176]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:35:21.550615669 (21h59m52.740914257s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:21.550621910 (21h59m52.740920502s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.698212587 (21h59m52.888511173s) [upload=21176]: Done uploading segments... Jul 25 09:35:21.698237000 (21h59m52.888535586s) [upload=21176]: Upload finished. err: <nil> Jul 25 09:35:50.802823283 (22h0m21.993121863s) [upload=21181]: Starting upload Jul 25 09:35:50.802897121 (22h0m21.993195707s) [upload=21181]: Done starting upload Jul 25 09:35:50.803220257 (22h0m21.993518825s) [upload=21181]: Uploading segments... Jul 25 09:35:50.803284411 (22h0m21.993582999s) [upload=21181]: Got next segment. Inline: false Jul 25 09:35:50.803442141 (22h0m21.993740722s) [upload=21181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:50.803448051 (22h0m21.99374663s) [upload=21181]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:20.533625759 (22h0m51.723924328s) [upload=21181]: Next segment err: context deadline exceeded Jul 25 09:36:20.533795008 (22h0m51.724093581s) [upload=21181]: Done uploading segments... Jul 25 09:36:20.533816345 (22h0m51.724114918s) [upload=21181]: Upload finished. err: context deadline exceeded Duration:* 2.400681993s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 3.132396808s FAIL (2025-07-25 10:07:08 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 21h59m10.08413493s) [seg_pos={0 0}, upload=21166]: Done begin upload segment. Jul 25 09:34:38.893861575 (21h59m10.084160157s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893877564 (21h59m10.084176144s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893970156 (21h59m10.08426874s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894325201 (21h59m10.084623773s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894381124 (21h59m10.084679703s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894891519 (21h59m10.085190109s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895339175 (21h59m10.085637757s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895721362 (21h59m10.086019941s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896080111 (21h59m10.086378691s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896408696 (21h59m10.086707281s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896758554 (21h59m10.087057132s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897033623 (21h59m10.087332202s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897322453 (21h59m10.087621038s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897596601 (21h59m10.087895181s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897880857 (21h59m10.088179437s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898303843 (21h59m10.088602412s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898661671 (21h59m10.08896025s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.899092762 (21h59m10.089391347s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:39.465168779 (21h59m10.655467365s) [upload=21166]: Next returned nil segment Jul 25 09:34:39.465194393 (21h59m10.655492983s) [upload=21166]: Waiting for error group managing segments... Jul 25 09:34:39.598220252 (21h59m10.788518839s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.604836354 (21h59m10.795134975s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.612452239 (21h59m10.80275081s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.619515598 (21h59m10.809814179s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.626845628 (21h59m10.817144209s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.627851431 (21h59m10.818150011s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.629266997 (21h59m10.819565577s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.645882287 (21h59m10.836180867s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.646651873 (21h59m10.836950453s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.647387132 (21h59m10.837685716s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.653519594 (21h59m10.843818177s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.660641895 (21h59m10.850940478s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.711883970 (21h59m10.902182542s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761622344 (21h59m10.951920924s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761651494 (21h59m10.951950073s) [seg_pos={0 0}, upload=21166]: Segment reached optimal threshold of 14 pieces. Jul 25 09:34:39.762107894 (21h59m10.952406474s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:39.762263719 (21h59m10.952562302s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qfp6gac6HdZquSvy2ouWCkMKieFDAN4JUQXg1zHSD3bNCBveV): protocol: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection Jul 25 09:34:39.762288380 (21h59m10.952586956s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TLFJ7HD7rVyLCyaKdy7DAJ1XvdN6s6gg6FuKQ58oJsRV6fhU6): protocol: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection Jul 25 09:34:39.762335180 (21h59m10.952633759s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vyNSWnc7efrHbDqsTmoeuMqg5JrS9dCA1sxEK4J3SfgJV5D5H): protocol: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection Jul 25 09:34:39.762356667 (21h59m10.952655237s) [seg_pos={0 0}, upload=21166]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:34:39.762441343 (21h59m10.952739924s) [upload=21166]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:39.762447146 (21h59m10.952745714s) [upload=21166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:39.911218699 (21h59m11.101517278s) [upload=21166]: Done uploading segments... Jul 25 09:34:39.911245243 (21h59m11.101543814s) [upload=21166]: Upload finished. err: <nil> Duration: 3.132425927s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.28928001s FAIL (2025-07-25 10:07:07 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.289314943s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.125396758s FAIL (2025-07-25 10:07:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== g segments... Jul 25 09:26:35.456904650 (21h51m6.64720323s) [upload=21038]: Upload finished. err: <nil> Jul 25 09:27:16.407968928 (21h51m47.598267507s) [upload=21052]: Starting upload Jul 25 09:27:16.408035705 (21h51m47.598334284s) [upload=21052]: Done starting upload Jul 25 09:27:16.408102508 (21h51m47.598401094s) [upload=21052]: Uploading segments... Jul 25 09:27:16.408182298 (21h51m47.598480891s) [upload=21052]: Got next segment. Inline: true Jul 25 09:27:16.408191016 (21h51m47.598489606s) [upload=21052]: Waiting for error group managing segments... Jul 25 09:27:16.408248128 (21h51m47.598546719s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:16.408254857 (21h51m47.598553447s) [upload=21052]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:16.408258740 (21h51m47.598557329s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:16.559578877 (21h51m47.749877447s) [upload=21052]: Done uploading segments... Jul 25 09:27:16.559612194 (21h51m47.749910766s) [upload=21052]: Upload finished. err: <nil> Jul 25 09:28:22.726097098 (21h52m53.916395667s) [upload=21069]: Starting upload Jul 25 09:28:22.726157005 (21h52m53.916455584s) [upload=21069]: Done starting upload Jul 25 09:28:22.726224882 (21h52m53.916523453s) [upload=21069]: Uploading segments... Jul 25 09:28:22.726267961 (21h52m53.916566549s) [upload=21069]: Got next segment. Inline: true Jul 25 09:28:22.726273573 (21h52m53.916572153s) [upload=21069]: Waiting for error group managing segments... Jul 25 09:28:22.726315230 (21h52m53.916613799s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:28:22.726320796 (21h52m53.916619376s) [upload=21069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:28:22.726324173 (21h52m53.916622752s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:28:22.871287048 (21h52m54.06158563s) [upload=21069]: Done uploading segments... Jul 25 09:28:22.871329296 (21h52m54.061627861s) [upload=21069]: Upload finished. err: <nil> Jul 25 09:29:03.680613055 (21h53m34.870911643s) [upload=21081]: Starting upload Jul 25 09:29:03.680698874 (21h53m34.870997445s) [upload=21081]: Done starting upload Jul 25 09:29:03.680794201 (21h53m34.871092783s) [upload=21081]: Uploading segments... Jul 25 09:29:03.680886067 (21h53m34.871184656s) [upload=21081]: Got next segment. Inline: true Jul 25 09:29:03.680906479 (21h53m34.871205059s) [upload=21081]: Waiting for error group managing segments... Jul 25 09:29:03.680962954 (21h53m34.871261535s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:03.680983505 (21h53m34.871282086s) [upload=21081]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:03.680987759 (21h53m34.871286339s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:03.928190102 (21h53m35.118488693s) [upload=21081]: Done uploading segments... Jul 25 09:29:03.928229672 (21h53m35.118528253s) [upload=21081]: Upload finished. err: <nil> Jul 25 09:29:18.766794416 (21h53m49.957092995s) [upload=21086]: Starting upload Jul 25 09:29:18.766860836 (21h53m49.957159459s) [upload=21086]: Done starting upload Jul 25 09:29:18.766928337 (21h53m49.957226924s) [upload=21086]: Uploading segments... Jul 25 09:29:18.767019576 (21h53m49.957318171s) [upload=21086]: Got next segment. Inline: true Jul 25 09:29:18.767028054 (21h53m49.957326646s) [upload=21086]: Waiting for error group managing segments... Jul 25 09:29:18.767082446 (21h53m49.957381032s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:18.767090462 (21h53m49.957389052s) [upload=21086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:18.767095196 (21h53m49.957393783s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:18.966677798 (21h53m50.156976378s) [upload=21086]: Done uploading segments... Jul 25 09:29:18.966694951 (21h53m50.156993524s) [upload=21086]: Upload finished. err: <nil> Jul 25 09:29:49.549110233 (21h54m20.739408822s) [upload=21096]: Starting upload Jul 25 09:29:49.549220943 (21h54m20.739519524s) [upload=21096]: Done starting upload Jul 25 09:29:49.549345183 (21h54m20.739643764s) [upload=21096]: Uploading segments... Jul 25 09:29:49.549394356 (21h54m20.739692958s) [upload=21096]: Got next segment. Inline: true Jul 25 09:29:49.549418287 (21h54m20.73971687s) [upload=21096]: Waiting for error group managing segments... Jul 25 09:29:49.549472215 (21h54m20.739770799s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:49.549491968 (21h54m20.739790549s) [upload=21096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:49.549499684 (21h54m20.739798268s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:49.712461693 (21h54m20.902760273s) [upload=21096]: Done uploading segments... Jul 25 09:29:49.712482595 (21h54m20.902781168s) [upload=21096]: Upload finished. err: <nil> Jul 25 09:30:51.603710658 (21h55m22.794009232s) [upload=21110]: Starting upload Jul 25 09:30:51.603792182 (21h55m22.794090761s) [upload=21110]: Done starting upload Jul 25 09:30:51.603874857 (21h55m22.79417343s) [upload=21110]: Uploading segments... Jul 25 09:30:51.603956850 (21h55m22.794255438s) [upload=21110]: Got next segment. Inline: true Jul 25 09:30:51.603963619 (21h55m22.794262189s) [upload=21110]: Waiting for error group managing segments... Jul 25 09:30:51.604025234 (21h55m22.794323804s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:51.604033463 (21h55m22.794332042s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:51.604038242 (21h55m22.794336822s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:51.761349757 (21h55m22.951648343s) [upload=21110]: Done uploading segments... Jul 25 09:30:51.761389314 (21h55m22.951687896s) [upload=21110]: Upload finished. err: <nil> Jul 25 09:31:17.167122347 (21h55m48.357420933s) [upload=21116]: Starting upload Jul 25 09:31:17.167244983 (21h55m48.357543565s) [upload=21116]: Done starting upload Jul 25 09:31:17.167355538 (21h55m48.357654109s) [upload=21116]: Uploading segments... Jul 25 09:31:17.167422267 (21h55m48.357720857s) [upload=21116]: Got next segment. Inline: true Jul 25 09:31:17.167428785 (21h55m48.357727375s) [upload=21116]: Waiting for error group managing segments... Jul 25 09:31:17.167501643 (21h55m48.357800233s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:17.167510622 (21h55m48.357809211s) [upload=21116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:17.167514153 (21h55m48.357812739s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:17.312004979 (21h55m48.502303564s) [upload=21116]: Done uploading segments... Jul 25 09:31:17.312029374 (21h55m48.502327955s) [upload=21116]: Upload finished. err: <nil> Jul 25 09:32:18.672367175 (21h56m49.862665762s) [upload=21134]: Starting upload Jul 25 09:32:18.672474129 (21h56m49.862772717s) [upload=21134]: Done starting upload Jul 25 09:32:18.672562251 (21h56m49.862860823s) [upload=21134]: Uploading segments... Jul 25 09:32:18.672624817 (21h56m49.862923407s) [upload=21134]: Got next segment. Inline: true Jul 25 09:32:18.672632560 (21h56m49.862931142s) [upload=21134]: Waiting for error group managing segments... Jul 25 09:32:18.672697593 (21h56m49.862996181s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:18.672705554 (21h56m49.86300414s) [upload=21134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:18.672710898 (21h56m49.863009481s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:18.921028738 (21h56m50.111327308s) [upload=21134]: Done uploading segments... Jul 25 09:32:18.921081796 (21h56m50.111380367s) [upload=21134]: Upload finished. err: <nil> Jul 25 09:33:18.245242378 (21h57m49.435540949s) [upload=21145]: Starting upload Jul 25 09:33:18.245313844 (21h57m49.435612415s) [upload=21145]: Done starting upload Jul 25 09:33:18.245389534 (21h57m49.435688106s) [upload=21145]: Uploading segments... Jul 25 09:33:18.245438781 (21h57m49.435737368s) [upload=21145]: Got next segment. Inline: true Jul 25 09:33:18.245444345 (21h57m49.435742915s) [upload=21145]: Waiting for error group managing segments... Jul 25 09:33:18.245508889 (21h57m49.435807471s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:18.245515616 (21h57m49.435814196s) [upload=21145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:18.245520233 (21h57m49.435818808s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:18.386658911 (21h57m49.576957489s) [upload=21145]: Done uploading segments... Jul 25 09:33:18.386751449 (21h57m49.577050032s) [upload=21145]: Upload finished. err: <nil> Jul 25 09:34:10.847242359 (21h58m42.037540934s) [upload=21161]: Starting upload Jul 25 09:34:10.847362462 (21h58m42.037661045s) [upload=21161]: Done starting upload Jul 25 09:34:10.847445757 (21h58m42.037744327s) [upload=21161]: Uploading segments... Jul 25 09:34:10.847489423 (21h58m42.037788008s) [upload=21161]: Got next segment. Inline: true Jul 25 09:34:10.847496016 (21h58m42.037794596s) [upload=21161]: Waiting for error group managing segments... Jul 25 09:34:10.847557787 (21h58m42.037856367s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:10.847563412 (21h58m42.037861992s) [upload=21161]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:34:10.847566753 (21h58m42.037865332s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:10.997618415 (21h58m42.187916989s) [upload=21161]: Done uploading segments... Jul 25 09:34:10.997644278 (21h58m42.187942846s) [upload=21161]: Upload finished. err: <nil> Duration: 1.125426131s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.063107023s FAIL (2025-07-25 10:07:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 403, RequestID: 185574E8772EFA87, HostID: , api error AccessDenied: Access Denied. Duration: 1.063136699s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................XX...............................X.........] duration: 945.273453ms FAIL (2025-07-25 10:06:58 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 945.367322ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 895.323901ms FAIL (2025-07-25 10:06:50 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 895.370608ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.284518919s FAIL (2025-07-25 10:06:47 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.284554937s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.373135968s FAIL (2025-07-25 10:06:42 UTC) orbiter warning: ======ERROR====== uplink: permission denied (bucket: metaclient: Unauthorized API credentials) ======LOGS====== 21h59m10.08413493s) [seg_pos={0 0}, upload=21166]: Done begin upload segment. Jul 25 09:34:38.893861575 (21h59m10.084160157s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893877564 (21h59m10.084176144s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.893970156 (21h59m10.08426874s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894325201 (21h59m10.084623773s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894381124 (21h59m10.084679703s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.894891519 (21h59m10.085190109s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895339175 (21h59m10.085637757s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.895721362 (21h59m10.086019941s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896080111 (21h59m10.086378691s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896408696 (21h59m10.086707281s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.896758554 (21h59m10.087057132s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897033623 (21h59m10.087332202s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897322453 (21h59m10.087621038s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897596601 (21h59m10.087895181s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.897880857 (21h59m10.088179437s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898303843 (21h59m10.088602412s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.898661671 (21h59m10.08896025s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:38.899092762 (21h59m10.089391347s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Uploading piece... Jul 25 09:34:39.465168779 (21h59m10.655467365s) [upload=21166]: Next returned nil segment Jul 25 09:34:39.465194393 (21h59m10.655492983s) [upload=21166]: Waiting for error group managing segments... Jul 25 09:34:39.598220252 (21h59m10.788518839s) [noise=true , address=77.90.31.12:4623 , piece_id=EJJLG5C4EQGF3YPKV5ASTZ5BJLFHWBNGTYZAED6Z2SPQUNYYXE6A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.604836354 (21h59m10.795134975s) [noise=true , address=77.90.31.13:4648 , piece_id=G42L6ATRDDXI74MLPUOJPRIY44CMMDSS7ZTBYTCOKTD24JVYQ2HA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.612452239 (21h59m10.80275081s) [noise=true , address=45.91.116.4:4959 , piece_id=OXWQJS53AUVFAWNCHC7MSTKW7EMJSXHQ7LADPJKRXJ25ASPXNU6Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.619515598 (21h59m10.809814179s) [noise=true , address=45.91.116.14:4206 , piece_id=4DKDNDE6SVRXV4LAB2KFZ5XTTYNBEJLFWYNRE24Q4RK3HJQ726VA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.626845628 (21h59m10.817144209s) [noise=true , address=45.91.116.12:4236 , piece_id=OYV3OJ5XXZ7L4PUQC3D4DYVSLMG7IYUXYEF6CR5Q6XSKFFEQMEYQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.627851431 (21h59m10.818150011s) [noise=true , address=77.90.31.14:4677 , piece_id=BBLTIOKPDFUSAQU5QJP4OYHHS3DUWAF4A7JCVYTMIK2T4AYRKTFQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.629266997 (21h59m10.819565577s) [noise=true , address=77.90.31.11:4575 , piece_id=B453ZLTT6WB4BTDN5PXPOCJVWJGZD7X5X56ULORGWZAEZ5LFQXXA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.645882287 (21h59m10.836180867s) [noise=true , address=77.90.31.17:4777 , piece_id=X5JOLEQHVA6OIP5QCKYGNWRBFP3ROZFXNZM3MOEXJS6VVVR5UU2A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.646651873 (21h59m10.836950453s) [noise=true , address=45.91.116.5:4330 , piece_id=LUY2KOA5WMDDTCNZO3PDLCFUA7AIIT5ELHY3NIHLH64FJNRDMDKQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.647387132 (21h59m10.837685716s) [noise=true , address=104.36.82.47:4519 , piece_id=O5B3VMADKURH7EMECDFQN6SGNAEQZPYQJIN6QM5NCF6ASG3LJLEA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.653519594 (21h59m10.843818177s) [noise=true , address=104.36.82.53:4155 , piece_id=VKS4AJ6U2EAE4GA7BA7AKKDPP7JTORYHDHAAZYXHOYUMUBJJYC3A, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.660641895 (21h59m10.850940478s) [noise=true , address=79.127.219.51:4213 , piece_id=D2EU7ROMTYMQFP4O4DECKTFUCD7TELGDD2PDK37QXCRMS52GXM2Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.711883970 (21h59m10.902182542s) [noise=true , address=79.127.219.38:4010 , piece_id=FD6OYCMF4EPO3V6F47RLBQMN43FA6QJUU6OAHL7AV53LW72LFU5Q, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761622344 (21h59m10.951920924s) [noise=true , address=79.127.220.101:4043 , piece_id=SOTH76HDYJXW4TH4FA7YPFASDO2G4ZQ34W5RGSSTV4K6ZFF6EWFA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: <nil> Jul 25 09:34:39.761651494 (21h59m10.951950073s) [seg_pos={0 0}, upload=21166]: Segment reached optimal threshold of 14 pieces. Jul 25 09:34:39.762107894 (21h59m10.952406474s) [noise=true , address=104.36.82.10:4100 , piece_id=DUE4HJ77XOF3XTMJGF3GSHZUOYYHRFAEJPDH6VGBVHV42HWT5NOA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW): protocol: expected piece hash; context canceled; EOF Jul 25 09:34:39.762263719 (21h59m10.952562302s) [noise=true , address=79.127.205.235:4673 , piece_id=LQFILV3IKB6UX4F3DO4SMIQSVOZCCP6KCFPOFGB2GXXBAJAETOUQ, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qfp6gac6HdZquSvy2ouWCkMKieFDAN4JUQXg1zHSD3bNCBveV): protocol: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60374->79.127.205.235:4673: use of closed network connection Jul 25 09:34:39.762288380 (21h59m10.952586956s) [noise=true , address=79.127.205.246:4272 , piece_id=WC2VYBPCCB5KS3F4XSY3RY5NH7MY2RWKSC7ASX6EIOCUSGEY54KA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TLFJ7HD7rVyLCyaKdy7DAJ1XvdN6s6gg6FuKQ58oJsRV6fhU6): protocol: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52154->79.127.205.246:4272: use of closed network connection Jul 25 09:34:39.762335180 (21h59m10.952633759s) [noise=true , address=79.127.205.245:4246 , piece_id=G7AG7DBLKOX2J7PDP6VEQTDFEHTW2E73QQ3WDIFZ7HTSS7L6XPTA, seg_pos={0 0}, upload=21166]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vyNSWnc7efrHbDqsTmoeuMqg5JrS9dCA1sxEK4J3SfgJV5D5H): protocol: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45568->79.127.205.245:4246: use of closed network connection Jul 25 09:34:39.762356667 (21h59m10.952655237s) [seg_pos={0 0}, upload=21166]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 25 09:34:39.762441343 (21h59m10.952739924s) [upload=21166]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:34:39.762447146 (21h59m10.952745714s) [upload=21166]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:39.911218699 (21h59m11.101517278s) [upload=21166]: Done uploading segments... Jul 25 09:34:39.911245243 (21h59m11.101543814s) [upload=21166]: Upload finished. err: <nil> Duration: 2.373163599s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 2.326873327s FAIL (2025-07-25 10:06:28 UTC) orbiter warning: ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== .BatchRequestItem_SegmentCommit Jul 25 09:34:40.505238896 (21h59m11.695537465s) [upload=21167]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:40.640979664 (21h59m11.831278264s) [upload=21167]: Done uploading segments... Jul 25 09:34:40.641009518 (21h59m11.831308091s) [upload=21167]: Upload finished. err: <nil> Jul 25 09:35:20.645390120 (21h59m51.835688706s) [upload=21176]: Starting upload Jul 25 09:35:20.645482113 (21h59m51.835780745s) [upload=21176]: Done starting upload Jul 25 09:35:20.645573260 (21h59m51.835871853s) [upload=21176]: Uploading segments... Jul 25 09:35:20.645792457 (21h59m51.836091041s) [upload=21176]: Got next segment. Inline: false Jul 25 09:35:20.645858035 (21h59m51.836156629s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:20.645865683 (21h59m51.836164267s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:35:20.754561036 (21h59m51.944859625s) [seg_pos={0 0}, upload=21176]: Begin upload segment... Jul 25 09:35:20.754647616 (21h59m51.944946206s) [seg_pos={0 0}, upload=21176]: Done begin upload segment. Jul 25 09:35:20.754787435 (21h59m51.945086024s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755353972 (21h59m51.94565256s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.755471416 (21h59m51.945770007s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756211019 (21h59m51.946509609s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.756667323 (21h59m51.946965913s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757099361 (21h59m51.947397948s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757508228 (21h59m51.947806818s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.757953278 (21h59m51.948251872s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758367934 (21h59m51.948666523s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758390422 (21h59m51.948689017s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.758878977 (21h59m51.949177567s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759244103 (21h59m51.949542684s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759545484 (21h59m51.949844064s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.759826303 (21h59m51.950124885s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760136389 (21h59m51.950434969s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:20.760432093 (21h59m51.950730676s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Uploading piece... Jul 25 09:35:21.372736599 (21h59m52.563035189s) [upload=21176]: Next returned nil segment Jul 25 09:35:21.372764315 (21h59m52.563062907s) [upload=21176]: Waiting for error group managing segments... Jul 25 09:35:21.511712315 (21h59m52.7020109s) [noise=true , address=45.91.116.6:4975 , piece_id=YHEAUCESV2UBMIVH5NAAB3WE3G3MPC3DNWLQOMOU3R6KMIHQAKGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512001132 (21h59m52.702299711s) [noise=true , address=45.91.116.11:4117 , piece_id=GDBGKTKIJMTRND35HPVI3MZTHJRWHTU47NGFNSTHL4M5FKFD6XEA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.512309738 (21h59m52.702608319s) [noise=true , address=45.91.116.7:4011 , piece_id=VBGYG4IKTYI753LGWKUK2MF6O6CTENREJO3CBGHPU6LM52BKUVBA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.517592678 (21h59m52.707891266s) [noise=true , address=45.91.116.4:4937 , piece_id=VZ6WLDPL3XW7VBRPTNNLZEGZYTRMUTEIH3OJKQFAHW4K6MYFURYA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.519619038 (21h59m52.709917619s) [noise=true , address=45.91.116.10:4083 , piece_id=QR5MKRQJ7NCGOI22D4FXIZ3NDNJHELNJKG5WZEBVUHEBLOKNMTWQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.520794731 (21h59m52.711093299s) [noise=true , address=45.91.116.14:4189 , piece_id=ZZOTO3VXXXHCFHJHVRN5LUT47O2JLX2CWQSVCBGDR43OFCR7JEYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.524808060 (21h59m52.715106643s) [noise=true , address=77.90.31.10:4520 , piece_id=K4WHWS35LH6Q6QYNQOAQYAS6V5MW2D5VHB2BHB3IUDFSEUL3635A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.527441251 (21h59m52.717739819s) [noise=true , address=45.91.116.8:4045 , piece_id=IFVAFWJRHQVITN34DY2WLV2RS5ON6QO3VAUF275TRPOCKYKLGMGQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.530273697 (21h59m52.720572278s) [noise=true , address=45.91.116.12:4227 , piece_id=IR27OBSDNRUJPGFIU7VKA4WLHX4C4K6Z7ZFPW5OOD6WH5RMR43RQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.540222474 (21h59m52.730521062s) [noise=true , address=77.90.31.14:4662 , piece_id=E45DXV4ZY6TVOYJRFW3PDCKUIIS2V4FYTLN5IV2SWRPVC7Q2ICIA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.541124873 (21h59m52.731423464s) [noise=true , address=104.36.82.2:4729 , piece_id=H6X4LEAMEMLGDTM7CYYQFL3PIU5RD2662XMDBMTTFFXNCTO4UZYQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.543682927 (21h59m52.733981514s) [noise=true , address=104.36.82.8:4969 , piece_id=XPK7RYOSQMFZFZHGQZPLJQSL2S7AP4MXWUB247MDB64EHCY2CHQA, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549663332 (21h59m52.739961915s) [noise=true , address=104.36.82.12:4149 , piece_id=JW4DKULNTICV4J3OCX6WLIZR6AXUOP6UFAVFREY4HWSLZ7MHJD5A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.549702269 (21h59m52.740000854s) [seg_pos={0 0}, upload=21176]: Segment reached optimal threshold of 13 pieces. Jul 25 09:35:21.549890135 (21h59m52.740188729s) [noise=true , address=104.36.82.7:4910 , piece_id=VLR5WZX7R4LYIRSSVGGNX5MYJRG3LBEV2DXJWUC2K4QARVMRDP3A, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550054494 (21h59m52.740353094s) [noise=true , address=104.36.82.11:4114 , piece_id=2ZLDFQGZSOLEZO5ENOGAHMOARRWEISWHLIHLDGMDZ2J6WVEVMUHQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Jul 25 09:35:21.550454434 (21h59m52.740753023s) [noise=true , address=104.36.82.16:4294 , piece_id=3EBQ2UNC7D6BGGX4AEPPU3CHUNLXHAIDN6O6JPW5UDW5Q4BOI7MQ, seg_pos={0 0}, upload=21176]: Done uploading piece. err: <nil> Jul 25 09:35:21.550506380 (21h59m52.740804969s) [seg_pos={0 0}, upload=21176]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Jul 25 09:35:21.550615669 (21h59m52.740914257s) [upload=21176]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 25 09:35:21.550621910 (21h59m52.740920502s) [upload=21176]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:35:21.698212587 (21h59m52.888511173s) [upload=21176]: Done uploading segments... Jul 25 09:35:21.698237000 (21h59m52.888535586s) [upload=21176]: Upload finished. err: <nil> Jul 25 09:35:50.802823283 (22h0m21.993121863s) [upload=21181]: Starting upload Jul 25 09:35:50.802897121 (22h0m21.993195707s) [upload=21181]: Done starting upload Jul 25 09:35:50.803220257 (22h0m21.993518825s) [upload=21181]: Uploading segments... Jul 25 09:35:50.803284411 (22h0m21.993582999s) [upload=21181]: Got next segment. Inline: false Jul 25 09:35:50.803442141 (22h0m21.993740722s) [upload=21181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:35:50.803448051 (22h0m21.99374663s) [upload=21181]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 25 09:36:20.533625759 (22h0m51.723924328s) [upload=21181]: Next segment err: context deadline exceeded Jul 25 09:36:20.533795008 (22h0m51.724093581s) [upload=21181]: Done uploading segments... Jul 25 09:36:20.533816345 (22h0m51.724114918s) [upload=21181]: Upload finished. err: context deadline exceeded Duration:* 2.32690606s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 892.114472ms FAIL (2025-07-25 10:06:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== g segments... Jul 25 09:26:35.456904650 (21h51m6.64720323s) [upload=21038]: Upload finished. err: <nil> Jul 25 09:27:16.407968928 (21h51m47.598267507s) [upload=21052]: Starting upload Jul 25 09:27:16.408035705 (21h51m47.598334284s) [upload=21052]: Done starting upload Jul 25 09:27:16.408102508 (21h51m47.598401094s) [upload=21052]: Uploading segments... Jul 25 09:27:16.408182298 (21h51m47.598480891s) [upload=21052]: Got next segment. Inline: true Jul 25 09:27:16.408191016 (21h51m47.598489606s) [upload=21052]: Waiting for error group managing segments... Jul 25 09:27:16.408248128 (21h51m47.598546719s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:27:16.408254857 (21h51m47.598553447s) [upload=21052]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:27:16.408258740 (21h51m47.598557329s) [upload=21052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:27:16.559578877 (21h51m47.749877447s) [upload=21052]: Done uploading segments... Jul 25 09:27:16.559612194 (21h51m47.749910766s) [upload=21052]: Upload finished. err: <nil> Jul 25 09:28:22.726097098 (21h52m53.916395667s) [upload=21069]: Starting upload Jul 25 09:28:22.726157005 (21h52m53.916455584s) [upload=21069]: Done starting upload Jul 25 09:28:22.726224882 (21h52m53.916523453s) [upload=21069]: Uploading segments... Jul 25 09:28:22.726267961 (21h52m53.916566549s) [upload=21069]: Got next segment. Inline: true Jul 25 09:28:22.726273573 (21h52m53.916572153s) [upload=21069]: Waiting for error group managing segments... Jul 25 09:28:22.726315230 (21h52m53.916613799s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:28:22.726320796 (21h52m53.916619376s) [upload=21069]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:28:22.726324173 (21h52m53.916622752s) [upload=21069]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:28:22.871287048 (21h52m54.06158563s) [upload=21069]: Done uploading segments... Jul 25 09:28:22.871329296 (21h52m54.061627861s) [upload=21069]: Upload finished. err: <nil> Jul 25 09:29:03.680613055 (21h53m34.870911643s) [upload=21081]: Starting upload Jul 25 09:29:03.680698874 (21h53m34.870997445s) [upload=21081]: Done starting upload Jul 25 09:29:03.680794201 (21h53m34.871092783s) [upload=21081]: Uploading segments... Jul 25 09:29:03.680886067 (21h53m34.871184656s) [upload=21081]: Got next segment. Inline: true Jul 25 09:29:03.680906479 (21h53m34.871205059s) [upload=21081]: Waiting for error group managing segments... Jul 25 09:29:03.680962954 (21h53m34.871261535s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:03.680983505 (21h53m34.871282086s) [upload=21081]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:03.680987759 (21h53m34.871286339s) [upload=21081]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:03.928190102 (21h53m35.118488693s) [upload=21081]: Done uploading segments... Jul 25 09:29:03.928229672 (21h53m35.118528253s) [upload=21081]: Upload finished. err: <nil> Jul 25 09:29:18.766794416 (21h53m49.957092995s) [upload=21086]: Starting upload Jul 25 09:29:18.766860836 (21h53m49.957159459s) [upload=21086]: Done starting upload Jul 25 09:29:18.766928337 (21h53m49.957226924s) [upload=21086]: Uploading segments... Jul 25 09:29:18.767019576 (21h53m49.957318171s) [upload=21086]: Got next segment. Inline: true Jul 25 09:29:18.767028054 (21h53m49.957326646s) [upload=21086]: Waiting for error group managing segments... Jul 25 09:29:18.767082446 (21h53m49.957381032s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:18.767090462 (21h53m49.957389052s) [upload=21086]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:18.767095196 (21h53m49.957393783s) [upload=21086]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:18.966677798 (21h53m50.156976378s) [upload=21086]: Done uploading segments... Jul 25 09:29:18.966694951 (21h53m50.156993524s) [upload=21086]: Upload finished. err: <nil> Jul 25 09:29:49.549110233 (21h54m20.739408822s) [upload=21096]: Starting upload Jul 25 09:29:49.549220943 (21h54m20.739519524s) [upload=21096]: Done starting upload Jul 25 09:29:49.549345183 (21h54m20.739643764s) [upload=21096]: Uploading segments... Jul 25 09:29:49.549394356 (21h54m20.739692958s) [upload=21096]: Got next segment. Inline: true Jul 25 09:29:49.549418287 (21h54m20.73971687s) [upload=21096]: Waiting for error group managing segments... Jul 25 09:29:49.549472215 (21h54m20.739770799s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:29:49.549491968 (21h54m20.739790549s) [upload=21096]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:29:49.549499684 (21h54m20.739798268s) [upload=21096]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:29:49.712461693 (21h54m20.902760273s) [upload=21096]: Done uploading segments... Jul 25 09:29:49.712482595 (21h54m20.902781168s) [upload=21096]: Upload finished. err: <nil> Jul 25 09:30:51.603710658 (21h55m22.794009232s) [upload=21110]: Starting upload Jul 25 09:30:51.603792182 (21h55m22.794090761s) [upload=21110]: Done starting upload Jul 25 09:30:51.603874857 (21h55m22.79417343s) [upload=21110]: Uploading segments... Jul 25 09:30:51.603956850 (21h55m22.794255438s) [upload=21110]: Got next segment. Inline: true Jul 25 09:30:51.603963619 (21h55m22.794262189s) [upload=21110]: Waiting for error group managing segments... Jul 25 09:30:51.604025234 (21h55m22.794323804s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:30:51.604033463 (21h55m22.794332042s) [upload=21110]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:30:51.604038242 (21h55m22.794336822s) [upload=21110]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:30:51.761349757 (21h55m22.951648343s) [upload=21110]: Done uploading segments... Jul 25 09:30:51.761389314 (21h55m22.951687896s) [upload=21110]: Upload finished. err: <nil> Jul 25 09:31:17.167122347 (21h55m48.357420933s) [upload=21116]: Starting upload Jul 25 09:31:17.167244983 (21h55m48.357543565s) [upload=21116]: Done starting upload Jul 25 09:31:17.167355538 (21h55m48.357654109s) [upload=21116]: Uploading segments... Jul 25 09:31:17.167422267 (21h55m48.357720857s) [upload=21116]: Got next segment. Inline: true Jul 25 09:31:17.167428785 (21h55m48.357727375s) [upload=21116]: Waiting for error group managing segments... Jul 25 09:31:17.167501643 (21h55m48.357800233s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:31:17.167510622 (21h55m48.357809211s) [upload=21116]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:31:17.167514153 (21h55m48.357812739s) [upload=21116]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:31:17.312004979 (21h55m48.502303564s) [upload=21116]: Done uploading segments... Jul 25 09:31:17.312029374 (21h55m48.502327955s) [upload=21116]: Upload finished. err: <nil> Jul 25 09:32:18.672367175 (21h56m49.862665762s) [upload=21134]: Starting upload Jul 25 09:32:18.672474129 (21h56m49.862772717s) [upload=21134]: Done starting upload Jul 25 09:32:18.672562251 (21h56m49.862860823s) [upload=21134]: Uploading segments... Jul 25 09:32:18.672624817 (21h56m49.862923407s) [upload=21134]: Got next segment. Inline: true Jul 25 09:32:18.672632560 (21h56m49.862931142s) [upload=21134]: Waiting for error group managing segments... Jul 25 09:32:18.672697593 (21h56m49.862996181s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:32:18.672705554 (21h56m49.86300414s) [upload=21134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:32:18.672710898 (21h56m49.863009481s) [upload=21134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:32:18.921028738 (21h56m50.111327308s) [upload=21134]: Done uploading segments... Jul 25 09:32:18.921081796 (21h56m50.111380367s) [upload=21134]: Upload finished. err: <nil> Jul 25 09:33:18.245242378 (21h57m49.435540949s) [upload=21145]: Starting upload Jul 25 09:33:18.245313844 (21h57m49.435612415s) [upload=21145]: Done starting upload Jul 25 09:33:18.245389534 (21h57m49.435688106s) [upload=21145]: Uploading segments... Jul 25 09:33:18.245438781 (21h57m49.435737368s) [upload=21145]: Got next segment. Inline: true Jul 25 09:33:18.245444345 (21h57m49.435742915s) [upload=21145]: Waiting for error group managing segments... Jul 25 09:33:18.245508889 (21h57m49.435807471s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:33:18.245515616 (21h57m49.435814196s) [upload=21145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:33:18.245520233 (21h57m49.435818808s) [upload=21145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:33:18.386658911 (21h57m49.576957489s) [upload=21145]: Done uploading segments... Jul 25 09:33:18.386751449 (21h57m49.577050032s) [upload=21145]: Upload finished. err: <nil> Jul 25 09:34:10.847242359 (21h58m42.037540934s) [upload=21161]: Starting upload Jul 25 09:34:10.847362462 (21h58m42.037661045s) [upload=21161]: Done starting upload Jul 25 09:34:10.847445757 (21h58m42.037744327s) [upload=21161]: Uploading segments... Jul 25 09:34:10.847489423 (21h58m42.037788008s) [upload=21161]: Got next segment. Inline: true Jul 25 09:34:10.847496016 (21h58m42.037794596s) [upload=21161]: Waiting for error group managing segments... Jul 25 09:34:10.847557787 (21h58m42.037856367s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 25 09:34:10.847563412 (21h58m42.037861992s) [upload=21161]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 25 09:34:10.847566753 (21h58m42.037865332s) [upload=21161]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 25 09:34:10.997618415 (21h58m42.187916989s) [upload=21161]: Done uploading segments... Jul 25 09:34:10.997644278 (21h58m42.187942846s) [upload=21161]: Upload finished. err: <nil> Duration: 892.145721ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................XX...............................X........] duration: 922.240619ms FAIL (2025-07-25 10:06:19 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 922.26743ms core uplink us1 vivint
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 325.859042ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Request-Id=DBm2nRVQhvj Accept-Ranges=bytes Content-Length=279 Server=Storj X-Amz-Request-Id=185574E527952D7A X-Xss-Protection=1; mode=block Date=Fri, 25 Jul 2025 09:36:16 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-07-25 10:06:16 UTC) orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185574E527952D7A</RequestId><HostId></HostId></Error>" Duration: 325.883263ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X.....XX......................................] duration: 2.272986948s FAIL (2025-07-25 10:06:10 UTC) orbiter warning: uplink: permission denied (bucket: metaclient: Unauthorized API credentials) Duration: 2.273015713s core uplink us1 vivint

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)