Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [..........XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX..................] duration:
3.115972094s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:45:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.116017438s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
3.016133548s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:44:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.016162459s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [...........XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.................] duration:
3.185108465s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:44:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.185137723s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.XXXX.......................................................] duration:
1.728625029s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:44:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.7286788s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
3.046161595s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:44:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.04621329s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..XXXX......................................................] duration:
1.704674571s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:43:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.704710827s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration:
2.877840051s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:43:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.877947881s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX................] duration:
3.334268123s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:43:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.334318192s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...XXXX.....................................................] duration:
1.581049769s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:42:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.581084137s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......XX....................................................] duration:
2.844704033s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:42:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.844734518s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......XX...................................................] duration:
2.896333611s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:42:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.896383735s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX...............] duration:
2.896769333s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:42:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.89680296s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [....XXXX....................................................] duration:
1.645571807s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:42:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.645671684s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [..............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX..............] duration:
3.164112277s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:41:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.164137288s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.....XXXX...................................................] duration:
2.188905695s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:41:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.188936129s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........XX..................................................] duration:
2.737905962s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:41:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.7379365s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [...............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.............] duration:
2.974526912s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:41:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.974554804s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX............] duration:
3.340340606s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:41:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.340368357s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [......XXXX..................................................] duration:
1.903531802s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:40:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.903558762s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........XX.................................................] duration:
2.911078863s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:40:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.911253834s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX...........] duration:
3.192921493s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:40:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.192958211s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.......XXXX.................................................] duration:
1.675311147s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:40:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.67533645s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........XX................................................] duration:
2.974681263s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:39:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.974709318s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...........XX...............................................] duration:
2.903926175s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:39:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.903953633s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [........XXXX................................................] duration:
2.157204771s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:39:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.157232862s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [..................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX..........] duration:
3.10684087s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:39:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.10688888s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [............XX..............................................] duration:
2.869045559s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:39:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.869072769s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.........XXXX...............................................] duration:
1.710157327s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:38:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.710186417s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [...................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.........] duration:
3.154529695s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:38:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.15456088s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..........XXXX..............................................] duration:
1.617943158s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:38:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.617984583s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.............XX.............................................] duration:
3.318456157s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:38:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.318480913s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [....................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX........] duration:
3.69336601s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:37:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.693418031s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...........XXXX.............................................] duration:
1.65240683s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:37:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.652432033s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.....................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.......] duration:
3.414165478s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:37:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.414202327s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..............XX............................................] duration:
2.831592758s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:37:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.831624295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [............XXXX............................................] duration:
1.724824408s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:36:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.724845319s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [......................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX......] duration:
2.769071742s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:36:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.769167677s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.......................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.....] duration:
3.346779723s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:36:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.346830303s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............XX...........................................] duration:
2.743430967s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:36:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.743452445s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.............XXXX...........................................] duration:
2.07578773s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:36:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.075807871s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX....] duration:
3.469260364s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:36:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.469285989s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................XX..........................................] duration:
3.368969635s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:35:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.368997449s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX...] duration:
26.943803676s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:35:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 26.943827198s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................XX.........................................] duration:
3.113124275s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:35:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.11315231s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..............XXXX..........................................] duration:
1.639906284s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:34:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.639934667s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [..........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX..] duration:
3.644115246s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:34:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.644143208s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................XX........................................] duration:
3.545506679s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:34:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.545539494s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [...........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.] duration:
3.192269584s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:34:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.19230028s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [............................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX] duration:
1m0.100467555s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:34:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1m0.100484437s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...............XXXX.........................................] duration:
1.60321044s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:33:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.603234786s
core
uplink
us-select-1
us1-placement-20
|