Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
306.827173ms remote:
"tcp:45.38.24.2:443" response headers:
X-Amz-Request-Id=1896F300521674B5 Date=Mon, 23 Feb 2026 18:18:06 GMT Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=XLWBrQsrQDY X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/325052e713be5cb5
FAIL (2026-02-23 18:18:06 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F300521674B5</RequestId><HostId></HostId></Error>"
Duration: 306.851982ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
9 out of 60 is failed (threshold=6) [XXX...XX....XX.X.................................X..........] duration:
1.627960208s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:18:05 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.628002714s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.714847ms remote:
"tcp:185.244.226.3:443" response headers:
X-Amz-Request-Id=1896F2FDCE6553C1 X-Request-Id=KEmJPfXk8Di Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:17:55 GMT Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/2dbb37454df8f492
FAIL (2026-02-23 18:17:55 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2FDCE6553C1</RequestId><HostId></HostId></Error>"
Duration: 9.736042ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
325.459893ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin Content-Length=284 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=1896F2FC26F12E3B X-Request-Id=6utz5FMcg1i X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:17:48 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/5c5885f3d7d8a078
FAIL (2026-02-23 18:17:48 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2FC26F12E3B</RequestId><HostId></HostId></Error>"
Duration: 325.489507ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.713931ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1896F2FB0BF65582 X-Request-Id=A9N8NGHv44x X-Xss-Protection=1; mode=block Accept-Ranges=bytes Date=Mon, 23 Feb 2026 18:17:43 GMT Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/3be44d546b39b4d3
FAIL (2026-02-23 18:17:43 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2FB0BF65582</RequestId><HostId></HostId></Error>"
Duration: 8.736095ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.417903502s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:17:41 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.417935171s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.495132ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1896F2F93B31FD9B Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=4sz3zAhYhFF X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:17:35 GMT Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/3f36066385278d58
FAIL (2026-02-23 18:17:35 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2F93B31FD9B</RequestId><HostId></HostId></Error>"
Duration: 8.519881ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
9 out of 60 is failed (threshold=6) [.XXX...XX....XX.X.................................X.........] duration:
2.133006411s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:17:35 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.133075699s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
906.843035ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:17:34 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 906.871184ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.001258ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Vary=Origin X-Amz-Request-Id=1896F2F5FAF7EE24 X-Request-Id=Np6vdWMqJcm X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Date=Mon, 23 Feb 2026 18:17:21 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/4784a806f7e807ef
FAIL (2026-02-23 18:17:21 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2F5FAF7EE24</RequestId><HostId></HostId></Error>"
Duration: 12.035766ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.2501ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Request-Id=8AaVGmN7jmG Date=Mon, 23 Feb 2026 18:16:57 GMT Content-Length=274 Server=Storj Vary=Origin X-Amz-Request-Id=1896F2F043F911E5 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/2356088a21e03f0
FAIL (2026-02-23 18:16:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2F043F911E5</RequestId><HostId></HostId></Error>"
Duration: 8.272631ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
10 out of 60 is failed (threshold=6) [X.XXX...XX....XX.X.................................X........] duration:
1.681871466s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:16:57 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.681895042s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.327169144s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:16:55 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.327193936s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.682783ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=dPboQ3eqBDJ X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:16:55 GMT Content-Length=274 X-Amz-Request-Id=1896F2EFEF368B35 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/1dc7215e28e66f7a
FAIL (2026-02-23 18:16:55 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2EFEF368B35</RequestId><HostId></HostId></Error>"
Duration: 8.719238ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
320.951104ms remote:
"tcp:45.38.24.2:443" response headers:
Date=Mon, 23 Feb 2026 18:16:47 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=1896F2EE00B9712C X-Request-Id=TMus7N1EGVU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/10d821eaefc5c9af
FAIL (2026-02-23 18:16:47 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2EE00B9712C</RequestId><HostId></HostId></Error>"
Duration: 320.973746ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.447318ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1896F2ED33D20C8F X-Request-Id=MUa3iYTTcmV Date=Mon, 23 Feb 2026 18:16:44 GMT Content-Length=274 Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/415a27af025e3da1
FAIL (2026-02-23 18:16:44 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2ED33D20C8F</RequestId><HostId></HostId></Error>"
Duration: 8.473888ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.379194247s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:16:38 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.379218357s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
929.276659ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:16:37 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 929.304294ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.507346ms remote:
"tcp:185.244.226.3:443" response headers:
X-Amz-Request-Id=1896F2E9EC44B121 X-Request-Id=WarptY7i4Rb X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:16:29 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Accept-Ranges=bytes Content-Type=application/xml Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/654bd45a9430269
FAIL (2026-02-23 18:16:29 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2E9EC44B121</RequestId><HostId></HostId></Error>"
Duration: 13.598866ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
17.246904ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Length=274 Content-Type=application/xml Server=Storj X-Amz-Request-Id=1896F2E3D0F352C6 X-Request-Id=bVF42ouY3hq X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:16:03 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/3bca5fa74cf6a345
FAIL (2026-02-23 18:16:03 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2E3D0F352C6</RequestId><HostId></HostId></Error>"
Duration: 17.300401ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.812425ms remote:
"tcp:185.244.226.3:443" response headers:
X-Xss-Protection=1; mode=block Vary=Origin X-Amz-Request-Id=1896F2E3D7A63CA9 X-Request-Id=avGQcYszNzU Date=Mon, 23 Feb 2026 18:16:03 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/9c979890bcf534b
FAIL (2026-02-23 18:16:03 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2E3D7A63CA9</RequestId><HostId></HostId></Error>"
Duration: 8.845881ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [XX.XXX...XX....XX.X.................................X.......] duration:
1.700735286s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:15:58 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.700778897s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
7.421031ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1896F2E277A9162A X-Request-Id=WPabZLRHHeR X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:15:57 GMT Accept-Ranges=bytes Server=Storj Content-Length=274 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/3854e9d6f85533a2
FAIL (2026-02-23 18:15:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2E277A9162A</RequestId><HostId></HostId></Error>"
Duration: 7.442846ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
309.333225ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Length=284 Content-Type=application/xml X-Amz-Request-Id=1896F2DF35C5E75B X-Request-Id=Tft5jZxHG2p Date=Mon, 23 Feb 2026 18:15:43 GMT Accept-Ranges=bytes target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/43df79ba75e31f5b
FAIL (2026-02-23 18:15:43 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2DF35C5E75B</RequestId><HostId></HostId></Error>"
Duration: 309.357151ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
896.392441ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:15:34 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 896.439381ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.184582ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Request-Id=LTCbx1pmtpX Date=Mon, 23 Feb 2026 18:15:33 GMT Vary=Origin X-Amz-Request-Id=1896F2DCABAB8FE2 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/5063fc76f0fe1e8d
FAIL (2026-02-23 18:15:33 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2DCABAB8FE2</RequestId><HostId></HostId></Error>"
Duration: 8.206116ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.388022955s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:15:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.38804945s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.78259ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Xss-Protection=1; mode=block Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1896F2DAA68278D1 X-Request-Id=fCvRWhRwfsQ Date=Mon, 23 Feb 2026 18:15:24 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/31425506a298216b
FAIL (2026-02-23 18:15:24 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2DAA68278D1</RequestId><HostId></HostId></Error>"
Duration: 12.804147ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.718825ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1896F2D8BF3C8C67 X-Request-Id=ZDs85YJNong Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:15:16 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/445675497b51b82e
FAIL (2026-02-23 18:15:16 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2D8BF3C8C67</RequestId><HostId></HostId></Error>"
Duration: 8.746509ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
12 out of 60 is failed (threshold=6) [XXX.XXX...XX....XX.X.................................X......] duration:
2.322388691s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:15:13 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.322418542s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
943.819351ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:15:04 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 943.915435ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.691403ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1896F2D306B8B96A X-Xss-Protection=1; mode=block Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=79RPubNvgjG Date=Mon, 23 Feb 2026 18:14:51 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/286c161dd3dd719b
FAIL (2026-02-23 18:14:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2D306B8B96A</RequestId><HostId></HostId></Error>"
Duration: 8.719737ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
288.475838ms remote:
"tcp:45.38.24.2:443" response headers:
X-Amz-Request-Id=1896F2D25AD00D42 X-Request-Id=9LF1YLAwm7K X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:14:48 GMT Content-Length=284 Content-Type=application/xml Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/3e6ec87897536189
FAIL (2026-02-23 18:14:48 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2D25AD00D42</RequestId><HostId></HostId></Error>"
Duration: 288.493263ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.889913ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Type=application/xml X-Amz-Request-Id=1896F2D0037EA405 X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:14:38 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=PfiwNXa96WQ Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/77112a7dd37314a3
FAIL (2026-02-23 18:14:38 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2D0037EA405</RequestId><HostId></HostId></Error>"
Duration: 8.920809ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.174343ms remote:
"tcp:185.244.226.3:443" response headers:
X-Amz-Request-Id=1896F2CFD5580BF8 X-Request-Id=ADXmvjoBSaD X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj Date=Mon, 23 Feb 2026 18:14:37 GMT Content-Security-Policy=block-all-mixed-content Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/421283ab352e0c9a
FAIL (2026-02-23 18:14:37 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2CFD5580BF8</RequestId><HostId></HostId></Error>"
Duration: 13.199796ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
330.259992ms remote:
"tcp:45.38.24.2:443" response headers:
Vary=Origin X-Amz-Request-Id=1896F2CE80F0B48E Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Request-Id=JVPAgrW4557 X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:14:32 GMT Content-Length=284 Content-Type=application/xml Server=Storj target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/7eac993dcbf88b3c
FAIL (2026-02-23 18:14:32 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2CE80F0B48E</RequestId><HostId></HostId></Error>"
Duration: 330.320056ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.348649944s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:14:26 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.348676233s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
12 out of 60 is failed (threshold=6) [.XXX.XXX...XX....XX.X.................................X.....] duration:
1.63931615s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:14:19 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.639341025s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
323.129492ms remote:
"tcp:45.38.24.2:443" response headers:
Server=Storj Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1896F2CA60B7232F X-Request-Id=89bV4Pkqb6R X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:14:14 GMT Content-Length=284 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/2fb778f181596391
FAIL (2026-02-23 18:14:14 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2CA60B7232F</RequestId><HostId></HostId></Error>"
Duration: 323.15753ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
936.985146ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:14:01 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 937.042446ms
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
12 out of 60 is failed (threshold=6) [..XXX.XXX...XX....XX.X.................................X....] duration:
2.363787579s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:14:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 2.363812895s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.352088ms remote:
"tcp:185.244.226.3:443" response headers:
X-Amz-Request-Id=1896F2C4FA40C708 X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:13:51 GMT Accept-Ranges=bytes Server=Storj Vary=Origin X-Request-Id=b2RQ2KjhDLb Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/403ef16957b9f15c
FAIL (2026-02-23 18:13:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2C4FA40C708</RequestId><HostId></HostId></Error>"
Duration: 13.373634ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
944.62552ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:13:43 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 944.652004ms
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.352631294s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:13:28 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.352701115s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
7.820065ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=1896F2BF4796B345 X-Request-Id=CeRiPraAjRh Date=Mon, 23 Feb 2026 18:13:26 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/7f9f36573ab17776
FAIL (2026-02-23 18:13:26 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2BF4796B345</RequestId><HostId></HostId></Error>"
Duration: 7.848148ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
12 out of 60 is failed (threshold=6) [...XXX.XXX...XX....XX.X.................................X...] duration:
1.62494641s trace:
https://jaeger.dev.storj.io/trace/7f545383a83cbdcf
FAIL (2026-02-23 18:13:26 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 1.62497428s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
7.82768ms remote:
"tcp:185.244.226.3:443" response headers:
X-Amz-Request-Id=1896F2BEA338B9D4 X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:13:24 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Scj5HruUtDq Accept-Ranges=bytes Content-Length=274 Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/5664a6f863ce59a7
FAIL (2026-02-23 18:13:24 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2BEA338B9D4</RequestId><HostId></HostId></Error>"
Duration: 7.851293ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.34857ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Length=274 Server=Storj Vary=Origin X-Amz-Request-Id=1896F2BE97F04A9C X-Request-Id=L9TaLW9bmsZ Date=Mon, 23 Feb 2026 18:13:23 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/8658d16a4f5a9a1
FAIL (2026-02-23 18:13:23 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2BE97F04A9C</RequestId><HostId></HostId></Error>"
Duration: 8.373229ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
325.024944ms remote:
"tcp:45.38.24.2:443" response headers:
Server=Storj X-Amz-Request-Id=1896F2BD50F0967E Date=Mon, 23 Feb 2026 18:13:18 GMT Content-Length=284 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Request-Id=52FTidHbKhx X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Type=application/xml target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/705870ade5300c9f
FAIL (2026-02-23 18:13:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2BD50F0967E</RequestId><HostId></HostId></Error>"
Duration: 325.06199ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.443624601s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:13:10 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.443649187s
core
uplink
us-select-1
|