Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.542421ms remote:
"tcp:185.244.226.3:443" response headers:
Date=Mon, 23 Feb 2026 18:10:57 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Length=274 Content-Type=application/xml X-Amz-Request-Id=1896F29C768C61B2 X-Request-Id=gfgUf6hJmr8 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/4633a02fd7da4aa3
FAIL (2026-02-23 18:10:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F29C768C61B2</RequestId><HostId></HostId></Error>"
Duration: 11.572538ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
3.274332392s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:10:49 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 3.274390198s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
898.921537ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:10:42 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 898.95974ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.806257ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:10:42 GMT Content-Length=274 Server=Storj X-Amz-Request-Id=1896F299040E5DC3 X-Request-Id=babEqj5ijyq target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/387257134da6c450
FAIL (2026-02-23 18:10:42 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F299040E5DC3</RequestId><HostId></HostId></Error>"
Duration: 9.889425ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
309.437903ms remote:
"tcp:45.38.24.2:443" response headers:
Server=Storj Vary=Origin X-Request-Id=Kio3aym8ruD X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1896F2965F3F460C Date=Mon, 23 Feb 2026 18:10:31 GMT Accept-Ranges=bytes Content-Length=284 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/53d12c8b275a39d7
FAIL (2026-02-23 18:10:31 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2965F3F460C</RequestId><HostId></HostId></Error>"
Duration: 309.519942ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.116709ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj X-Amz-Request-Id=1896F2963E582DE7 Date=Mon, 23 Feb 2026 18:10:30 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Request-Id=cLK8JAQyj7h X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/5d212a9688189d1
FAIL (2026-02-23 18:10:30 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2963E582DE7</RequestId><HostId></HostId></Error>"
Duration: 9.142771ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.450327ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml X-Request-Id=MhCkJ2iygR9 X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=1896F295E7D72CAC Date=Mon, 23 Feb 2026 18:10:29 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/69d3dee5ead029c5
FAIL (2026-02-23 18:10:29 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F295E7D72CAC</RequestId><HostId></HostId></Error>"
Duration: 8.540168ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [.......XXX.XXX...XX....XX.X.................................] duration:
1.638999756s trace:
https://jaeger.dev.storj.io/trace/41620cd69692420b
FAIL (2026-02-23 18:10:21 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.639044796s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.875745ms remote:
"tcp:185.244.226.3:443" response headers:
X-Amz-Request-Id=1896F29321A23588 X-Request-Id=3V3voHu616R X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:10:17 GMT Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/38013e29f2e2153a
FAIL (2026-02-23 18:10:17 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F29321A23588</RequestId><HostId></HostId></Error>"
Duration: 8.901808ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.167436ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=1896F291E37620A5 X-Xss-Protection=1; mode=block Content-Type=application/xml X-Request-Id=NjDEHnCRWGb Date=Mon, 23 Feb 2026 18:10:11 GMT Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/391a47c8c747053c
FAIL (2026-02-23 18:10:11 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F291E37620A5</RequestId><HostId></HostId></Error>"
Duration: 13.239291ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.464214105s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:10:02 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.464312296s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.963423ms remote:
"tcp:185.244.226.3:443" response headers:
Date=Mon, 23 Feb 2026 18:09:55 GMT Server=Storj Vary=Origin X-Amz-Request-Id=1896F28E1ABCEE10 X-Request-Id=PZRRcoxvDSN Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/63109c74ee077d70
FAIL (2026-02-23 18:09:55 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F28E1ABCEE10</RequestId><HostId></HostId></Error>"
Duration: 9.994798ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
7.480427ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Dqpddja1uKH Server=Storj X-Amz-Request-Id=1896F28DF670D7FA X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:09:54 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/2e11f5a24fc27567
FAIL (2026-02-23 18:09:54 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F28DF670D7FA</RequestId><HostId></HostId></Error>"
Duration: 7.504007ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [........XXX.XXX...XX....XX.X................................] duration:
1.727621165s trace:
https://jaeger.dev.storj.io/trace/41620cd69692420b
FAIL (2026-02-23 18:09:40 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.727647561s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
985.521807ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:09:31 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 985.551901ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
307.346611ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=Rsuf2BBjDan X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 X-Amz-Request-Id=1896F287B9FF58CD Date=Mon, 23 Feb 2026 18:09:28 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/70e30a65237fb091
FAIL (2026-02-23 18:09:28 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F287B9FF58CD</RequestId><HostId></HostId></Error>"
Duration: 307.370172ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.621698ms remote:
"tcp:185.244.226.3:443" response headers:
Server=Storj Vary=Origin X-Amz-Request-Id=1896F286F300A6C2 X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:09:24 GMT Content-Length=274 Content-Type=application/xml X-Request-Id=BSXdUcGz5A4 Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/54e4d6a1a5ce5085
FAIL (2026-02-23 18:09:24 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F286F300A6C2</RequestId><HostId></HostId></Error>"
Duration: 12.648363ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.427134479s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:09:13 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.42716089s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [.........XXX.XXX...XX....XX.X...............................] duration:
2.116044797s trace:
https://jaeger.dev.storj.io/trace/41620cd69692420b
FAIL (2026-02-23 18:09:06 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.116078181s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
10.296676ms remote:
"tcp:185.244.226.3:443" response headers:
X-Amz-Request-Id=1896F28228748BF3 Date=Mon, 23 Feb 2026 18:09:04 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Request-Id=9iGuczGcD18 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/7f479e3bc377bd3b
FAIL (2026-02-23 18:09:04 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F28228748BF3</RequestId><HostId></HostId></Error>"
Duration: 10.341966ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.186895ms remote:
"tcp:185.244.226.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:08:58 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=1896F280B454A441 Accept-Ranges=bytes Vary=Origin X-Request-Id=Kx95KhNdGyq target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/58d33e3838c1a6e4
FAIL (2026-02-23 18:08:58 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F280B454A441</RequestId><HostId></HostId></Error>"
Duration: 8.212588ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
954.520273ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:08:43 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 954.547966ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.713645ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml X-Amz-Request-Id=1896F27BEBBD44FE X-Request-Id=iw9uEp7zrej Date=Mon, 23 Feb 2026 18:08:37 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1c41bb0f77997ea9
FAIL (2026-02-23 18:08:37 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F27BEBBD44FE</RequestId><HostId></HostId></Error>"
Duration: 8.734688ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
7.002004ms remote:
"tcp:185.244.226.3:443" response headers:
X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Type=application/xml Vary=Origin X-Amz-Request-Id=1896F27BED1DCFFD X-Request-Id=4zmvL139eRp Date=Mon, 23 Feb 2026 18:08:37 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/1dc09b69a6285ab1
FAIL (2026-02-23 18:08:37 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F27BED1DCFFD</RequestId><HostId></HostId></Error>"
Duration: 7.028152ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
307.135033ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=5LTYc5WSWBX Content-Length=284 Server=Storj X-Amz-Request-Id=1896F27BA7BBA318 X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:08:36 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/24621750a120bdd1
FAIL (2026-02-23 18:08:36 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F27BA7BBA318</RequestId><HostId></HostId></Error>"
Duration: 307.160164ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
37.727099ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Accept-Ranges=bytes Server=Storj Vary=Origin X-Amz-Request-Id=1896F27A5C96F7D3 X-Request-Id=Uxfo9ssoZZL X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:08:30 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/120320287b8578f5
FAIL (2026-02-23 18:08:30 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F27A5C96F7D3</RequestId><HostId></HostId></Error>"
Duration: 37.752348ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.362498054s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:08:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.362523988s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [..........XXX.XXX...XX....XX.X..............................] duration:
1.813163195s trace:
https://jaeger.dev.storj.io/trace/41620cd69692420b
FAIL (2026-02-23 18:08:07 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.813191062s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [...........XXX.XXX...XX....XX.X.............................] duration:
1.633851879s trace:
https://jaeger.dev.storj.io/trace/41620cd69692420b
FAIL (2026-02-23 18:07:51 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.633878174s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.623318ms remote:
"tcp:185.244.226.3:443" response headers:
Date=Mon, 23 Feb 2026 18:07:50 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=ifcNYrBN553 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1896F27117B42772 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/58b813e0d3b0662d
FAIL (2026-02-23 18:07:50 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F27117B42772</RequestId><HostId></HostId></Error>"
Duration: 8.648634ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.98083ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1896F26FB7D9912C Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content X-Request-Id=aNUPkL4mkRZ X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:07:45 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/39f5a6f942b54c92
FAIL (2026-02-23 18:07:45 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F26FB7D9912C</RequestId><HostId></HostId></Error>"
Duration: 10.009826ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
42.542271ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Vary=Origin X-Amz-Request-Id=1896F26F4B181574 X-Request-Id=8VHa9a5m58K Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:07:43 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/517739ae34ba1888
FAIL (2026-02-23 18:07:43 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F26F4B181574</RequestId><HostId></HostId></Error>"
Duration: 42.568271ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
900.188945ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:07:39 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 900.223429ms
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [............XXX.XXX...XX....XX.X............................] duration:
1.569855233s trace:
https://jaeger.dev.storj.io/trace/41620cd69692420b
FAIL (2026-02-23 18:07:24 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.569880972s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
290.132777ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Length=284 Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=1896F26AC6BE68BB X-Request-Id=YtiRbo5tuq5 Date=Mon, 23 Feb 2026 18:07:23 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/4707a9a0c4746aba
FAIL (2026-02-23 18:07:23 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F26AC6BE68BB</RequestId><HostId></HostId></Error>"
Duration: 290.156675ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.377826266s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:07:12 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.377910255s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.954199ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=1896F267B8979139 Accept-Ranges=bytes Vary=Origin X-Request-Id=MT4uZeAZNgy X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:07:10 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/5f37c93dccd88019
FAIL (2026-02-23 18:07:10 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F267B8979139</RequestId><HostId></HostId></Error>"
Duration: 14.983405ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
7.767235ms remote:
"tcp:185.244.226.3:443" response headers:
X-Request-Id=fFygWotDdSt Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:07:05 GMT Content-Length=274 Content-Type=application/xml Server=Storj X-Amz-Request-Id=1896F26688E3CE79 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/54c642f923af8462
FAIL (2026-02-23 18:07:05 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F26688E3CE79</RequestId><HostId></HostId></Error>"
Duration: 7.798885ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [.............XXX.XXX...XX....XX.X...........................] duration:
1.780353947s trace:
https://jaeger.dev.storj.io/trace/41620cd69692420b
FAIL (2026-02-23 18:07:03 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.780464409s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.885069ms remote:
"tcp:185.244.226.3:443" response headers:
Content-Length=274 Content-Type=application/xml Server=Storj X-Amz-Request-Id=1896F2646D9F09F6 X-Request-Id=hxuJaxXDQv1 Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:06:56 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/2c997b0b20969384
FAIL (2026-02-23 18:06:56 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2646D9F09F6</RequestId><HostId></HostId></Error>"
Duration: 8.907281ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.309202125s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:06:44 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.30922619s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
911.207741ms trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:06:43 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 911.251202ms
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
42.406993ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Date=Mon, 23 Feb 2026 18:06:34 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1896F25F59B83B26 X-Request-Id=Mo2JD3fmdx8 X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/43fded3174a6d0d9
FAIL (2026-02-23 18:06:34 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F25F59B83B26</RequestId><HostId></HostId></Error>"
Duration: 42.423323ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.478673388s trace:
https://jaeger.dev.storj.io/trace/36e97d8015f22124
FAIL (2026-02-23 18:06:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
uplink: metainfo: metaclient: context deadline exceeded
======LOGS======
Feb 23 17:41:56.004070144 (141h25m0.938524289s) [upload=124341]: Starting upload
Feb 23 17:41:56.004173173 (141h25m0.938627315s) [upload=124341]: Done starting upload
Feb 23 17:41:56.004286224 (141h25m0.93874038s) [upload=124341]: Uploading segments...
Feb 23 17:41:56.004613148 (141h25m0.93906727s) [upload=124341]: Got next segment. Inline: false
Feb 23 17:41:56.004773516 (141h25m0.93922766s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 23 17:41:56.004782051 (141h25m0.939236194s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 23 17:41:56.963639957 (141h25m1.898094089s) [seg_pos={0 0}, upload=124341]: Begin upload segment...
Feb 23 17:41:56.963724470 (141h25m1.898178608s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964806857 (141h25m1.89926098s) [seg_pos={0 0}, upload=124341]: Done begin upload segment.
Feb 23 17:41:56.964962744 (141h25m1.899416883s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.964990880 (141h25m1.899445024s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965451429 (141h25m1.899905571s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965586288 (141h25m1.900040427s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.965847678 (141h25m1.900301813s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966118833 (141h25m1.900572974s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966384902 (141h25m1.900839041s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.966580263 (141h25m1.901034401s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967025032 (141h25m1.901479185s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.967646573 (141h25m1.902100717s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968168297 (141h25m1.902622435s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.968721971 (141h25m1.903176112s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970081793 (141h25m1.904535931s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.970615369 (141h25m1.905069508s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:56.971157004 (141h25m1.905611143s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Uploading piece...
Feb 23 17:41:57.608094655 (141h25m2.542548778s) [upload=124341]: Next returned nil segment
Feb 23 17:41:57.608112892 (141h25m2.542567029s) [upload=124341]: Waiting for error group managing segments...
Feb 23 17:41:57.742822687 (141h25m2.67727682s) [noise=true , address=145.79.227.33:4872 , piece_id=LUNVRFWVAXHKKSPK4OZTOVDBY6F7AWND4KNALLUJL642QGPXBYKA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.749456751 (141h25m2.683910874s) [noise=true , address=145.79.227.51:4412 , piece_id=FAAGEOKHTBN4OAK7KVXQOLE2PJQBSUTV6IJMCG656CJ4WEFJU3FA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750137124 (141h25m2.684591248s) [noise=true , address=145.79.227.6:4260 , piece_id=OFZI7KHTDXJVJPTTGF326U5WAKR372DMNSJLBHKLQ5KTOAOSB2XA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.750500993 (141h25m2.68495512s) [noise=true , address=145.79.227.4:4186 , piece_id=47WUSFKH2DW5FSFIX5L6O3CXZNT3A6XXIZ5FRQLWLPRNIHZXQ2MA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752008284 (141h25m2.686462405s) [noise=true , address=145.79.227.29:4943 , piece_id=PY7GYW5IS5MEDLHF4KBUZNTGUPKO55SOGTNBELEGJNLP3N7I3DVA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.752538019 (141h25m2.686992152s) [noise=true , address=45.91.116.11:4120 , piece_id=AVDTFMT42T6NW3XAHV7SC4IRVN24PVW54MUJXJZ5QETMS22WKQEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.754819111 (141h25m2.689273243s) [noise=true , address=145.79.227.30:4767 , piece_id=EKJYGN57EEWKERPPZLNFSNCVXRE2H2L3DEKWQFHTFOSILW7EX7WQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.759963442 (141h25m2.694417584s) [noise=true , address=45.91.116.5:4332 , piece_id=P72RVOC2U4PF5AIBRYY3DSFHCEAX5BHCNT3THRQBHN3LHNFMSANA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.760607237 (141h25m2.695061369s) [noise=true , address=145.79.227.31:4803 , piece_id=MB42IHUXM523TWBQIYTDTKEGHWXZIWCGLU6GMLQQZ7EKMOR2DYJA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.761853193 (141h25m2.696307335s) [noise=true , address=145.79.227.7:4297 , piece_id=HKWBOAM226AL6OM5DQF4CNUPPOKY6XC2DGN6IUVBFOWQEROGHUVQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.763535570 (141h25m2.697989712s) [noise=true , address=145.79.227.22:4514 , piece_id=T4FFTDMKBIVD4WW3UHDR7OO2THKQ6BTBKMZBHDULYCRIV3QPTMOQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.764161527 (141h25m2.698615666s) [noise=true , address=145.79.227.28:4731 , piece_id=NP2Q53QFHVSCHCAVEP3PUDYTZDAMNIUGEZSIK3LQZ4H7A3ZNZX6Q, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778254265 (141h25m2.712708402s) [noise=true , address=145.79.227.47:4270 , piece_id=DJ3U7UBWMHND2GEPACZHOMOTJIZAHGAYLSVBGMCBV5PJSMHSNJCQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: <nil>
Feb 23 17:41:57.778285949 (141h25m2.712740091s) [seg_pos={0 0}, upload=124341]: Satisfied cohort requirements after 13 pieces
Feb 23 17:41:57.778570838 (141h25m2.713024981s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778582859 (141h25m2.713036996s) [noise=true , address=77.90.31.18:4808 , piece_id=RMEGCBZU2FWR7YBLS4DHGWYGKTDIVR53D6XKCLMXLJQKPMD3ZKEA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.778762950 (141h25m2.713217096s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QJBWV6xLXS7DcX8d26be79z1DH3r27oi4EcuiwVwR5Dn8JEGh): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.778779349 (141h25m2.713233492s) [noise=true , address=145.79.227.42:4121 , piece_id=CDZ3T627OOQ2TLXABKEWGZ7DPRAXXB4IVOBLH3PMRXKQIJ737VEQ, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779144382 (141h25m2.713598526s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pvLo4F5ieDtepopg8Wi8jw6PKNeDCDvtH5iVNQXBUQwSXYxuD): protocol: expected piece hash; context canceled; EOF
Feb 23 17:41:57.779157428 (141h25m2.713611571s) [noise=true , address=79.127.219.41:4928 , piece_id=HEIAL4K5IQG6VY7KTXGN6A2PU66HY3KZT5Z44U3IE67R6AJK56RA, seg_pos={0 0}, upload=124341]: Long tail cancellation due to optimal threshold
Feb 23 17:41:57.779216837 (141h25m2.713670959s) [seg_pos={0 0}, upload=124341]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Feb 23 17:41:57.779392538 (141h25m2.713846681s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Feb 23 17:41:57.779400435 (141h25m2.713854577s) [upload=124341]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Feb 23 17:42:01.523450339 (141h25m6.457904485s) [upload=124341]: Done uploading segments...
Feb 23 17:42:01.523492869 (141h25m6.457947008s) [upload=124341]: Upload finished. err: <nil>
Duration: 2.478701482s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.050667ms remote:
"tcp:185.244.226.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml X-Xss-Protection=1; mode=block Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=1896F25AB0A65801 X-Request-Id=REbxceih15u Date=Mon, 23 Feb 2026 18:06:14 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/725f048b89ba1194
FAIL (2026-02-23 18:06:14 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F25AB0A65801</RequestId><HostId></HostId></Error>"
Duration: 8.072877ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.816164ms remote:
"tcp:185.244.226.3:443" response headers:
Vary=Origin X-Amz-Request-Id=1896F25A11F29495 Date=Mon, 23 Feb 2026 18:06:12 GMT Content-Type=application/xml Server=Storj X-Request-Id=Yg6myKFXEzx X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/62fdce17c13c84c7
FAIL (2026-02-23 18:06:12 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F25A11F29495</RequestId><HostId></HostId></Error>"
Duration: 12.841467ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
322.387404ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Type=application/xml X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=1896F259A8CCA4B2 X-Request-Id=AWngLpHgucF Date=Mon, 23 Feb 2026 18:06:10 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/46ff9a54438b24bb
FAIL (2026-02-23 18:06:10 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F259A8CCA4B2</RequestId><HostId></HostId></Error>"
Duration: 322.41404ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.017986202s trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:06:08 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.018010644s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
7.537366ms remote:
"tcp:185.244.226.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Mon, 23 Feb 2026 18:05:59 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1896F2572287E8BA Content-Security-Policy=block-all-mixed-content X-Request-Id=AiAeZHyqa7g target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1e85763e20611be4
FAIL (2026-02-23 18:05:59 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1896F2572287E8BA</RequestId><HostId></HostId></Error>"
Duration: 7.56073ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
1.084744237s trace:
https://jaeger.dev.storj.io/trace/11507cad06aa5223
FAIL (2026-02-23 18:05:52 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 1.084832248s
core
uplink
us1
|