Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 468377 reports (Page 4266 of 9368)
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
8.092989ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Type=application/xml
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:08:25 GMT
Accept-Ranges=bytes
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE36CB19BFB6
X-Request-Id=D9H9uh6Dzoz
Content-Length=274
Content-Security-Policy=block-all-mixed-content
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/7d43beda67176c28
FAIL (2026-02-26 18:08:25 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE36CB19BFB6</RequestId><HostId></HostId></Error>"
Duration: 8.112978ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
2.390044215s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:08:23 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.390074415s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [..X.....X...........X..X..........XX........................]
duration:
1.544444661s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:08:18 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.544482683s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
27.263583ms
remote:
"tcp:185.244.226.3:443"
response headers:
X-Request-Id=8D745kXMbPp
X-Xss-Protection=1; mode=block
Content-Type=application/xml
Server=Storj
Date=Thu, 26 Feb 2026 18:08:12 GMT
Accept-Ranges=bytes
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Vary=Origin
X-Amz-Request-Id=1897DE33E1E70E1B
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
trace:
https://jaeger.dev.storj.io/trace/280cb31a67d1537
FAIL (2026-02-26 18:08:12 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE33E1E70E1B</RequestId><HostId></HostId></Error>"
Duration: 27.303422ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
290.064362ms
remote:
"tcp:45.38.24.2:443"
response headers:
Content-Type=application/xml
X-Request-Id=PdgADsWF8Z8
Content-Length=284
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE32BE8E2AF9
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:08:07 GMT
Accept-Ranges=bytes
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/5576317521695ed1
FAIL (2026-02-26 18:08:07 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE32BE8E2AF9</RequestId><HostId></HostId></Error>"
Duration: 290.09798ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
29.569813ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Length=274
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE3149D53613
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
X-Request-Id=ZRzoQfPSxfH
Date=Thu, 26 Feb 2026 18:08:01 GMT
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
trace:
https://jaeger.dev.storj.io/trace/555db6e1e7508ac4
FAIL (2026-02-26 18:08:01 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE3149D53613</RequestId><HostId></HostId></Error>"
Duration: 29.601072ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
2.36592482s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:07:33 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.365963418s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
7.99598ms
remote:
"tcp:185.244.226.3:443"
response headers:
Server=Storj
X-Request-Id=3kKHX4suwQR
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:07:24 GMT
Accept-Ranges=bytes
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Vary=Origin
X-Amz-Request-Id=1897DE289C64D0C1
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/1352f00d71f64e50
FAIL (2026-02-26 18:07:24 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE289C64D0C1</RequestId><HostId></HostId></Error>"
Duration: 8.02662ms edge gateway vhost bgp eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [...X.....X...........X..X..........XX.......................]
duration:
2.706363639s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:07:21 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.706393017s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
2.542985137s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:07:10 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.543034559s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
10.135071ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Type=application/xml
X-Amz-Request-Id=1897DE2519CF43EC
X-Request-Id=5QzTEtZAte4
X-Xss-Protection=1; mode=block
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
Date=Thu, 26 Feb 2026 18:07:09 GMT
Accept-Ranges=bytes
Content-Length=274
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/78819a809ee6f7bf
FAIL (2026-02-26 18:07:09 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE2519CF43EC</RequestId><HostId></HostId></Error>"
Duration: 10.151774ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
314.668513ms
remote:
"tcp:45.38.24.2:443"
response headers:
Vary=Origin
X-Amz-Request-Id=1897DE2362DEE9AC
X-Xss-Protection=1; mode=block
Content-Type=application/xml
Server=Storj
X-Request-Id=AyGke4h5Lnp
Date=Thu, 26 Feb 2026 18:07:01 GMT
Accept-Ranges=bytes
Content-Length=284
Content-Security-Policy=block-all-mixed-content
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/5e49070cd3168335
FAIL (2026-02-26 18:07:02 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE2362DEE9AC</RequestId><HostId></HostId></Error>"
Duration: 314.694137ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
7.768684ms
remote:
"tcp:185.244.226.3:443"
response headers:
X-Request-Id=ZgHGjo4pzvb
Accept-Ranges=bytes
Content-Length=274
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:06:58 GMT
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE227E05A58A
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
trace:
https://jaeger.dev.storj.io/trace/7bf4608cc88ffab7
FAIL (2026-02-26 18:06:58 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE227E05A58A</RequestId><HostId></HostId></Error>"
Duration: 7.792587ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
12.294786ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Server=Storj
X-Amz-Request-Id=1897DE21943D46D1
Content-Type=application/xml
Vary=Origin
X-Request-Id=Ue2r7LVdhZY
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:06:54 GMT
Accept-Ranges=bytes
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
trace:
https://jaeger.dev.storj.io/trace/5cff08478a7feb1a
FAIL (2026-02-26 18:06:54 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE21943D46D1</RequestId><HostId></HostId></Error>"
Duration: 12.390174ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
2.481126252s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:06:35 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.481238455s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [....X.....X...........X..X..........XX......................]
duration:
1.932406527s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:06:29 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.932438013s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
9.049926ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Security-Policy=block-all-mixed-content
Server=Storj
X-Request-Id=d3FysZ3DyJq
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Length=274
Content-Type=application/xml
Vary=Origin
X-Amz-Request-Id=1897DE185773B238
Date=Thu, 26 Feb 2026 18:06:14 GMT
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/3e6830fe17c5a043
FAIL (2026-02-26 18:06:14 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE185773B238</RequestId><HostId></HostId></Error>"
Duration: 9.091156ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
4.34943194s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:06:04 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 4.349465079s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
292.315394ms
remote:
"tcp:45.38.24.2:443"
response headers:
Content-Length=284
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE15477075FA
X-Request-Id=b8vucDfEBP1
Date=Thu, 26 Feb 2026 18:06:01 GMT
Content-Security-Policy=block-all-mixed-content
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/4167e7e9adc6a63b
FAIL (2026-02-26 18:06:01 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE15477075FA</RequestId><HostId></HostId></Error>"
Duration: 292.374889ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
20.475424ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE150A5E67B0
X-Request-Id=3RrNNYHRoCS
Date=Thu, 26 Feb 2026 18:06:00 GMT
Content-Type=application/xml
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Length=274
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
trace:
https://jaeger.dev.storj.io/trace/516f18dbb701260b
FAIL (2026-02-26 18:06:00 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE150A5E67B0</RequestId><HostId></HostId></Error>"
Duration: 20.502031ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
9.660822ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
X-Request-Id=4YABfwo1ddA
Accept-Ranges=bytes
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE11D9BD5D1D
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:05:46 GMT
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
trace:
https://jaeger.dev.storj.io/trace/587efde7e1c5cb98
FAIL (2026-02-26 18:05:46 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE11D9BD5D1D</RequestId><HostId></HostId></Error>"
Duration: 9.808585ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
6 out of 60 is failed (threshold=6) [.....X.....X...........X..X..........XX.....................]
duration:
1.679296266s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:05:38 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.679339684s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
14.464707ms
remote:
"tcp:185.244.226.3:443"
response headers:
X-Amz-Request-Id=1897DE0F75566587
Date=Thu, 26 Feb 2026 18:05:36 GMT
Content-Security-Policy=block-all-mixed-content
Server=Storj
X-Request-Id=GzehKhnnSq5
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Length=274
Content-Type=application/xml
Vary=Origin
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/7232a3da7c947792
FAIL (2026-02-26 18:05:36 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE0F75566587</RequestId><HostId></HostId></Error>"
Duration: 14.545019ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
8.412476ms
remote:
"tcp:185.244.226.3:443"
response headers:
X-Request-Id=J9XhM1H1rah
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:05:08 GMT
Accept-Ranges=bytes
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE08F8320F2B
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
trace:
https://jaeger.dev.storj.io/trace/569e3340cad39f5e
FAIL (2026-02-26 18:05:08 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE08F8320F2B</RequestId><HostId></HostId></Error>"
Duration: 8.456776ms edge gateway vhost bgp ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
2.390891995s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:05:00 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.39093097s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
304.486844ms
remote:
"tcp:45.38.24.2:443"
response headers:
Content-Length=284
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
Accept-Ranges=bytes
Content-Type=application/xml
X-Amz-Request-Id=1897DE06F93DEEBF
X-Request-Id=c7AUQv7ymNz
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:04:59 GMT
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/3075392e274b5316
FAIL (2026-02-26 18:04:59 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE06F93DEEBF</RequestId><HostId></HostId></Error>"
Duration: 304.516565ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
8.604484ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE05BDCE0EA4
Date=Thu, 26 Feb 2026 18:04:54 GMT
Accept-Ranges=bytes
Content-Length=274
Content-Security-Policy=block-all-mixed-content
X-Request-Id=6uzRzvnJZHK
X-Xss-Protection=1; mode=block
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/4e9ee3fcb1078aaa
FAIL (2026-02-26 18:04:54 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE05BDCE0EA4</RequestId><HostId></HostId></Error>"
Duration: 8.631205ms edge gateway vhost bgp eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
7 out of 60 is failed (threshold=6) [X.....X.....X...........X..X..........XX....................]
duration:
1.685269333s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:04:53 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.685299321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
2.418183667s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:04:45 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.418224932s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
288.115967ms
remote:
"tcp:45.38.24.2:443"
response headers:
Content-Length=284
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE01E2E96BA0
Date=Thu, 26 Feb 2026 18:04:38 GMT
Accept-Ranges=bytes
Content-Type=application/xml
X-Request-Id=7nQ9NSiJX2g
X-Xss-Protection=1; mode=block
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/8a10f9644de3d42
FAIL (2026-02-26 18:04:38 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE01E2E96BA0</RequestId><HostId></HostId></Error>"
Duration: 288.23769ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
7.58251ms
remote:
"tcp:185.244.226.3:443"
response headers:
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE00ED0344C6
X-Request-Id=6VyeQsSsVo8
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Type=application/xml
Date=Thu, 26 Feb 2026 18:04:33 GMT
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
trace:
https://jaeger.dev.storj.io/trace/e84166bc500d953
FAIL (2026-02-26 18:04:33 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE00ED0344C6</RequestId><HostId></HostId></Error>"
Duration: 7.639649ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
12.27198ms
remote:
"tcp:185.244.226.3:443"
response headers:
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:04:30 GMT
Accept-Ranges=bytes
Content-Security-Policy=block-all-mixed-content
X-Request-Id=NDL8f3p9bc
Content-Length=274
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DE0035660AD4
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
trace:
https://jaeger.dev.storj.io/trace/58ad44eada7b126
FAIL (2026-02-26 18:04:30 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DE0035660AD4</RequestId><HostId></HostId></Error>"
Duration: 12.302686ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
711.253975ms
remote:
"tcp:45.38.24.2:443"
response headers:
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DDFA02E85985
X-Xss-Protection=1; mode=block
X-Request-Id=fmwuhBnoZpr
Date=Thu, 26 Feb 2026 18:04:04 GMT
Accept-Ranges=bytes
Content-Length=284
Content-Security-Policy=block-all-mixed-content
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/1ad7a401cbea306c
FAIL (2026-02-26 18:04:04 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDFA02E85985</RequestId><HostId></HostId></Error>"
Duration: 711.291996ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
8.108005ms
remote:
"tcp:185.244.226.3:443"
response headers:
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DDF8930563AD
X-Request-Id=9LZAUQaXrDy
X-Xss-Protection=1; mode=block
Content-Security-Policy=block-all-mixed-content
Date=Thu, 26 Feb 2026 18:03:58 GMT
Accept-Ranges=bytes
Content-Length=274
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/4665539b5b5258b4
FAIL (2026-02-26 18:03:58 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDF8930563AD</RequestId><HostId></HostId></Error>"
Duration: 8.129239ms edge gateway vhost bgp eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
2.354382591s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:03:52 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.354413922s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
7 out of 60 is failed (threshold=6) [.X.....X.....X...........X..X..........XX...................]
duration:
1.617004297s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:03:50 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.617047276s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
289.385873ms
remote:
"tcp:45.38.24.2:443"
response headers:
Date=Thu, 26 Feb 2026 18:03:45 GMT
Accept-Ranges=bytes
X-Amz-Request-Id=1897DDF5BD03B9C6
Content-Length=284
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Request-Id=KWoyyhWeChy
X-Xss-Protection=1; mode=block
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/226134612715db76
FAIL (2026-02-26 18:03:45 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDF5BD03B9C6</RequestId><HostId></HostId></Error>"
Duration: 289.410772ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
8.360504ms
remote:
"tcp:185.244.226.3:443"
response headers:
X-Amz-Request-Id=1897DDF3A84397A8
Accept-Ranges=bytes
Server=Storj
X-Request-Id=Wfgi6UeY8dt
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:03:36 GMT
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Vary=Origin
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011
trace:
https://jaeger.dev.storj.io/trace/70e095c8b3149977
FAIL (2026-02-26 18:03:36 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDF3A84397A8</RequestId><HostId></HostId></Error>"
Duration: 8.385956ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
10.458689ms
remote:
"tcp:185.244.226.3:443"
response headers:
X-Amz-Request-Id=1897DDF29E0F6181
Date=Thu, 26 Feb 2026 18:03:32 GMT
Accept-Ranges=bytes
Content-Length=274
Server=Storj
Vary=Origin
X-Request-Id=Qiv5L2ue5Yb
X-Xss-Protection=1; mode=block
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
trace:
https://jaeger.dev.storj.io/trace/7dea06065f316138
FAIL (2026-02-26 18:03:32 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDF29E0F6181</RequestId><HostId></HostId></Error>"
Duration: 10.517798ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
12.814604ms
remote:
"tcp:185.244.226.3:443"
response headers:
Accept-Ranges=bytes
Content-Length=274
Content-Security-Policy=block-all-mixed-content
X-Amz-Request-Id=1897DDF230E298AA
Date=Thu, 26 Feb 2026 18:03:30 GMT
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Request-Id=D6ybeDQHFiv
X-Xss-Protection=1; mode=block
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/45ed9e7f63cea120
FAIL (2026-02-26 18:03:30 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDF230E298AA</RequestId><HostId></HostId></Error>"
Duration: 12.925616ms edge gateway vhost bgp eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
7 out of 60 is failed (threshold=6) [..X.....X.....X...........X..X..........XX..................]
duration:
2.345701007s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:03:25 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.345730324s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
2.750980902s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:03:12 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 2.751007158s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
30.664129339s
trace:
https://jaeger.dev.storj.io/trace/68a06934c0a05f04
FAIL (2026-02-26 18:03:09 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
eb 26 18:02:40.761285883 (213h45m45.695740025s) [upload=188070]: Starting upload
Feb 26 18:02:40.761395708 (213h45m45.695849854s) [upload=188070]: Done starting upload
Feb 26 18:02:40.761697240 (213h45m45.696151374s) [upload=188070]: Uploading segments...
Feb 26 18:02:40.762148293 (213h45m45.696602435s) [upload=188070]: Got next segment. Inline: false
Feb 26 18:02:40.762222848 (213h45m45.696676987s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 26 18:02:40.762230722 (213h45m45.69668487s) [upload=188070]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 26 18:02:41.270627701 (213h45m46.20508184s) [seg_pos={0 0}, upload=188070]: Begin upload segment...
Feb 26 18:02:41.270720334 (213h45m46.205174474s) [seg_pos={0 0}, upload=188070]: Done begin upload segment.
Feb 26 18:02:41.270802481 (213h45m46.205256658s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272042027 (213h45m46.206496159s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272283707 (213h45m46.20673785s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272744753 (213h45m46.207198908s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.272938814 (213h45m46.207392958s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273277379 (213h45m46.207731498s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273552846 (213h45m46.208006989s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.273597171 (213h45m46.208051303s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274079549 (213h45m46.208533671s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274139867 (213h45m46.208594009s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274541012 (213h45m46.208995213s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.274823659 (213h45m46.209277801s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.275440186 (213h45m46.209894339s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.276035884 (213h45m46.210490026s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.277403363 (213h45m46.211857506s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.279624637 (213h45m46.214078779s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Uploading piece...
Feb 26 18:02:41.889995128 (213h45m46.824449265s) [upload=188070]: Next returned nil segment
Feb 26 18:02:41.890012492 (213h45m46.824466626s) [upload=188070]: Waiting for error group managing segments...
Feb 26 18:02:42.025533148 (213h45m46.959987291s) [noise=true , address=45.91.116.14:4196    , piece_id=HCNADG4RROT5POVVKVRRLPG47GSJNTKKPN4VKZFZLRAQIA2AGQGA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.026652488 (213h45m46.961106618s) [noise=true , address=45.91.116.12:4260    , piece_id=2JAUJW744WIAF62XHFXYLVETLQKTPB7JPZWL2OYFOJJBHIQ4YEAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.067448882 (213h45m47.001903023s) [noise=true , address=145.79.227.27:4692   , piece_id=J3DCU4JTRRKRHXSITWSREXSLAC457LHYKH3WAKWY3676LQNRLBIQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.074637015 (213h45m47.009091156s) [noise=true , address=145.79.227.49:4340   , piece_id=4DY3PUAA3YK64RAGQJG7AQL7A4326N4JS3EPF2IWLXS374JFLD2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085003774 (213h45m47.019457898s) [noise=true , address=145.79.227.34:4909   , piece_id=FBBP5DIR225WLVV3ZOOINTY6V6BW4IUSU4XAWFMBGCPX2VJ5QNFA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.085343706 (213h45m47.019797831s) [noise=true , address=145.79.227.18:4007   , piece_id=B74HKCRWM5O4UBJHC4SUCTCCB234ZJXXKSRB7GDLLS2BA2KEKYAA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.129629735 (213h45m47.064083879s) [noise=true , address=145.79.227.7:4298    , piece_id=3MZOHCVKY6RJQFI3YBRQI4N7QD5HQOEQGMETMBSQVHZAOZPPCU2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:42.152804779 (213h45m47.087258916s) [noise=true , address=77.90.31.19:4842     , piece_id=LTJK4F4OQJIW3JP3QGX4RXWOW2RXQCGAUJV3PJSRQVWOYTSDQHVQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:55.018263081 (213h45m59.952717215s) [noise=true , address=145.79.227.28:4727   , piece_id=ANJU4GLFZ2CMCNUDPHKTBIPAQRU67QKHIRBBXTYEN2YKXNJFN5NA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:02:56.602166399 (213h46m1.536620544s) [noise=true , address=145.79.227.39:4013   , piece_id=HF6BOIHHMPC6SVI2KVBLOAK5PE5AUDASGPFV6XGX2EVBQF262R2Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:03.112945430 (213h46m8.047399567s) [noise=true , address=145.79.227.16:4911   , piece_id=R7XS2TLYUBGTWKTV7WYJASYGPIS4F2IIOIO6AM2O345OK2PVNR2A, seg_pos={0 0}, upload=188070]: Done uploading piece. err: <nil>
Feb 26 18:03:08.984934397 (213h46m13.919388539s) [noise=true , address=145.79.227.17:4975   , piece_id=WHAFQWHPO3P6CJXK2UTQIVUI5XW5AY4MVGPE6N6IEU2RNHMF7DHA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12WiUZeNCJk4rzjtyWuKJwGcZATaYLU2QxN27MZE7i45zVpu9dM, address:145.79.227.17:4975): protocol: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60550->145.79.227.17:4975: use of closed network connection
Feb 26 18:03:08.985159188 (213h46m13.919613328s) [noise=true , address=145.79.227.38:4982   , piece_id=CDJZME7DQXICXLJNZUGUG6TCT7MD7MOMIZLNPNGNW5RJLQ3E3Q4Q, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:1NxoYJXCXmTvQncbkXt6yH7mhEBBrzVdzhgL5J6W5TD4N3pkaG, address:145.79.227.38:4982): protocol: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34792->145.79.227.38:4982: use of closed network connection
Feb 26 18:03:08.985363612 (213h46m13.919817753s) [noise=true , address=145.79.227.24:4584   , piece_id=5WWPMU7BO3WLX5A6SAOYMFTO2YDVAAB6TLBVUBOUDAJMRT33OLYQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: expected piece hash; context deadline exceeded; EOF
Feb 26 18:03:08.985393310 (213h46m13.919847443s) [noise=true , address=145.79.227.14:4840   , piece_id=4B562IFYNDJL25GTL5TVLYCAZDJCDWXUPSOW4UCRJH2SCJIXSLTA, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54136->145.79.227.14:4840: use of closed network connection
Feb 26 18:03:08.985453811 (213h46m13.919907936s) [noise=true , address=145.79.227.5:4224    , piece_id=3IBRVOY65JJUBM4KY4MINDAUU5OFPGDD5WQBWJAKMJODFSFI6IRQ, seg_pos={0 0}, upload=188070]: Done uploading piece. err: ecclient: upload failed (node:128nLffK44ioiTpjWjAm2vHFTLouLZdw7BijqBAbsdjo99kPYNo, address:145.79.227.5:4224): protocol: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:39750->145.79.227.5:4224: use of closed network connection
Feb 26 18:03:08.985487049 (213h46m13.919941172s) [seg_pos={0 0}, upload=188070]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Feb 26 18:03:09.647036800 (213h46m14.581490932s) [upload=188070]: Done uploading segments...
Feb 26 18:03:09.647104855 (213h46m14.581559003s) [upload=188070]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 30.664153488s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
7 out of 60 is failed (threshold=6) [...X.....X.....X...........X..X..........XX.................]
duration:
1.674111218s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:03:03 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.674129367s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
319.471687ms
remote:
"tcp:45.38.24.2:443"
response headers:
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:03:01 GMT
Accept-Ranges=bytes
Content-Length=284
Content-Type=application/xml
Server=Storj
Vary=Origin
Content-Security-Policy=block-all-mixed-content
X-Amz-Request-Id=1897DDEB4D490A75
X-Request-Id=UN2JsJbuMB8
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/697df50c124e029d
FAIL (2026-02-26 18:03:01 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDEB4D490A75</RequestId><HostId></HostId></Error>"
Duration: 319.515635ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
9.000195ms
remote:
"tcp:185.244.226.3:443"
response headers:
Accept-Ranges=bytes
Content-Length=274
Content-Type=application/xml
X-Request-Id=SomGZ7fGrHw
Date=Thu, 26 Feb 2026 18:02:55 GMT
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
X-Amz-Request-Id=1897DDE9F6E2CECF
X-Xss-Protection=1; mode=block
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/4b7bf36d350bfdfb
FAIL (2026-02-26 18:02:55 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDE9F6E2CECF</RequestId><HostId></HostId></Error>"
Duration: 9.025828ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
14.45602ms
remote:
"tcp:185.244.226.3:443"
response headers:
Accept-Ranges=bytes
Content-Length=274
Content-Type=application/xml
X-Amz-Request-Id=1897DDE723A87B3F
X-Request-Id=WeGwKrpbizR
Content-Security-Policy=block-all-mixed-content
Server=Storj
Vary=Origin
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:02:43 GMT
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
trace:
https://jaeger.dev.storj.io/trace/3fcbb09143e72d3a
FAIL (2026-02-26 18:02:43 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDE723A87B3F</RequestId><HostId></HostId></Error>"
Duration: 14.482876ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
304.914831ms
remote:
"tcp:45.38.24.2:443"
response headers:
X-Xss-Protection=1; mode=block
Date=Thu, 26 Feb 2026 18:02:39 GMT
Accept-Ranges=bytes
Server=Storj
X-Amz-Request-Id=1897DDE637D62760
X-Request-Id=yEHiWG2SZv
Content-Length=284
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Vary=Origin
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
trace:
https://jaeger.dev.storj.io/trace/559b778e5a482725
FAIL (2026-02-26 18:02:39 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDE637D62760</RequestId><HostId></HostId></Error>"
Duration: 304.945989ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
7 out of 60 is failed (threshold=6) [....X.....X.....X...........X..X..........XX................]
duration:
1.650349439s
trace:
https://jaeger.dev.storj.io/trace/3eba5f9dcfb45a38
FAIL (2026-02-26 18:02:32 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.65038033s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)
antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
9.586084ms
remote:
"tcp:185.244.226.3:443"
response headers:
Accept-Ranges=bytes
Content-Length=274
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Vary=Origin
X-Request-Id=bXS2ywHp45q
Date=Thu, 26 Feb 2026 18:02:31 GMT
Server=Storj
X-Amz-Request-Id=1897DDE47A92BFEC
X-Xss-Protection=1; mode=block
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318
trace:
https://jaeger.dev.storj.io/trace/2142cb39fcc0121f
FAIL (2026-02-26 18:02:31 UTC)
orbiter warning: unexpected status code "403 Forbidden"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897DDE47A92BFEC</RequestId><HostId></HostId></Error>"
Duration: 9.615584ms edge gateway vhost bgp eu1