Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..XXXX......................................................] duration:
1.704674571s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:43:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.704710827s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration:
2.877840051s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:43:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.877947881s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX................] duration:
3.334268123s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:43:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.334318192s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...XXXX.....................................................] duration:
1.581049769s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:42:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.581084137s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......XX....................................................] duration:
2.844704033s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:42:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.844734518s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......XX...................................................] duration:
2.896333611s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:42:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.896383735s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX...............] duration:
2.896769333s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:42:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.89680296s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [....XXXX....................................................] duration:
1.645571807s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:42:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.645671684s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [..............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX..............] duration:
3.164112277s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:41:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.164137288s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.....XXXX...................................................] duration:
2.188905695s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:41:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.188936129s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........XX..................................................] duration:
2.737905962s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:41:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.7379365s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [...............XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.............] duration:
2.974526912s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:41:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.974554804s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX............] duration:
3.340340606s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:41:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.340368357s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [......XXXX..................................................] duration:
1.903531802s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:40:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.903558762s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........XX.................................................] duration:
2.911078863s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:40:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.911253834s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX...........] duration:
3.192921493s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:40:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.192958211s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.......XXXX.................................................] duration:
1.675311147s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:40:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.67533645s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........XX................................................] duration:
2.974681263s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:39:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.974709318s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...........XX...............................................] duration:
2.903926175s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:39:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.903953633s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [........XXXX................................................] duration:
2.157204771s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:39:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.157232862s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [..................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX..........] duration:
3.10684087s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:39:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.10688888s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [............XX..............................................] duration:
2.869045559s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:39:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.869072769s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.........XXXX...............................................] duration:
1.710157327s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:38:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.710186417s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [...................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.........] duration:
3.154529695s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:38:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.15456088s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..........XXXX..............................................] duration:
1.617943158s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:38:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.617984583s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.............XX.............................................] duration:
3.318456157s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:38:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.318480913s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [....................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX........] duration:
3.69336601s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:37:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.693418031s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...........XXXX.............................................] duration:
1.65240683s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:37:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.652432033s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.....................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.......] duration:
3.414165478s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:37:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.414202327s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..............XX............................................] duration:
2.831592758s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:37:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.831624295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [............XXXX............................................] duration:
1.724824408s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:36:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.724845319s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [......................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX......] duration:
2.769071742s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:36:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 2.769167677s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.......................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.....] duration:
3.346779723s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:36:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.346830303s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............XX...........................................] duration:
2.743430967s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:36:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.743452445s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.............XXXX...........................................] duration:
2.07578773s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:36:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.075807871s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX....] duration:
3.469260364s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:36:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.469285989s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................XX..........................................] duration:
3.368969635s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:35:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.368997449s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [.........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX...] duration:
26.943803676s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:35:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 26.943827198s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................XX.........................................] duration:
3.113124275s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:35:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.11315231s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..............XXXX..........................................] duration:
1.639906284s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:34:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.639934667s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [..........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX..] duration:
3.644115246s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:34:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.644143208s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................XX........................................] duration:
3.545506679s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:34:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.545539494s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [...........................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX.] duration:
3.192269584s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:34:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 3.19230028s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
26 out of 60 is failed (threshold=3) [............................XX.XX...X.XXXXXXXXXXXXXXXXX.XXXX] duration:
1m0.100467555s trace:
https://jaeger.dev.storj.io/trace/7f908743098f0576
FAIL (2026-02-17 23:34:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
7261153 (3h16m15.601715291s) [upload=2852]: Got next segment. Inline: false
Feb 17 23:33:10.667314810 (3h16m15.601768947s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:33:10.667321427 (3h16m15.601775597s) [upload=2852]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:33:10.776607940 (3h16m15.71106208s) [seg_pos={0 0}, upload=2852]: Begin upload segment...
Feb 17 23:33:10.776696742 (3h16m15.711150866s) [seg_pos={0 0}, upload=2852]: Done begin upload segment.
Feb 17 23:33:10.776760025 (3h16m15.711214166s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778171620 (3h16m15.712625756s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778280870 (3h16m15.712735011s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778678541 (3h16m15.71313268s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.778792620 (3h16m15.713246763s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779139233 (3h16m15.713593373s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779180385 (3h16m15.713634522s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779530340 (3h16m15.713984463s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779548787 (3h16m15.714002929s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779875552 (3h16m15.714329686s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.779903904 (3h16m15.71435804s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780300286 (3h16m15.714754424s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.780855656 (3h16m15.715309795s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781278384 (3h16m15.715732543s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781392731 (3h16m15.715846873s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.781698983 (3h16m15.716153121s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.782036364 (3h16m15.716490502s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:10.783236453 (3h16m15.717690579s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Uploading piece...
Feb 17 23:33:11.325135194 (3h16m16.25958933s) [upload=2852]: Next returned nil segment
Feb 17 23:33:11.325180688 (3h16m16.259634829s) [upload=2852]: Waiting for error group managing segments...
Feb 17 23:33:11.478370676 (3h16m16.412824815s) [noise=true , address=145.79.227.46:4617 , piece_id=FOI4GGL26BC3SLDV6EBYJQUDAGV2KTUPO6H2ONHNHII3ASD7DBWQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.489702119 (3h16m16.424156258s) [noise=true , address=145.79.227.2:4148 , piece_id=AZPLYCDJZG6YCSVF74C7OKLQ7X72ACSB2LR56YLS54TAVCARKOPQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.492159746 (3h16m16.426613883s) [noise=true , address=45.91.116.13:4161 , piece_id=JNFCHBJQWF3ZH7MZOXIGGTU3TI7YDBLZGB2OT5GEMIVYSN7QZVDA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.520249445 (3h16m16.454703572s) [noise=true , address=145.79.227.5:4256 , piece_id=36DCUTC75KOJFGCSLCSYD5FFDE34NQMCQ3K33G7C7DBKKKIWUESQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.533706957 (3h16m16.46816108s) [noise=true , address=145.79.227.19:4101 , piece_id=TG7I4JAMR2UAI5H2A7VBLRJBD5MYFKB5XXZBOFJLKFJAYE5AKV6Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:11.586898275 (3h16m16.5213524s) [noise=true , address=145.79.227.41:4097 , piece_id=5576YCUAGDCKXMKUDVQXC2UTKSRRRIXWHYVNG47HDVMHLAYXZTSQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011356848 (3h16m16.945810981s) [noise=true , address=193.149.164.70:4107 , piece_id=JHXBFJBLSI4HQ4JGCIBBPZ4Q4AA3O5NX5C5UIC3C4IAE2FWZ7GZQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.011462569 (3h16m16.945916731s) [noise=true , address=193.149.164.69:4093 , piece_id=735ICYWJIZOKH3HBZ26KU6C2SJC6IVEMIX2BMGOLOILIVAZQZLYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.020460262 (3h16m16.954914395s) [noise=true , address=193.149.164.66:4983 , piece_id=LO4RHURY4HLD3W6YMFO6B7P7MUXCTGJZSPMUOWHY3AXNJ7EFBYZA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.025949884 (3h16m16.960404006s) [noise=true , address=193.149.164.67:4018 , piece_id=5VUO47PF4A2JQX2U323VHGL5ZQR6W3O3M2ECWYRFWWVMLBZEDRHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.039088110 (3h16m16.973542232s) [noise=true , address=64.49.13.140:4510 , piece_id=CIGWVYBMKE3MXBDU3MJGQENEXPRM43B3DMOVLIUCT6GJJ323UFHQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:33:12.425365568 (3h16m17.359819691s) [noise=true , address=145.79.227.30:4768 , piece_id=USGQQFK6T555AA2BGQFN3M3CRY4S2MB7YKCP7VWG3MREL2DFTJ2Q, seg_pos={0 0}, upload=2852]: Done uploading piece. err: <nil>
Feb 17 23:34:10.399361497 (3h17m15.333815691s) [noise=true , address=77.90.31.16:4750 , piece_id=YT7RXY626TDCW33K4TFBAYQ6HR4QG6RHMCTU6SUPHJITOITPP6PA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12TGd8NnWdekDtHzBQHmJPgTtHjXC6g8fi6RoZYFQJVHDa5hdyE, address:77.90.31.16:4750): protocol: rpc: dial tcp 77.90.31.16:4750: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399572798 (3h17m15.334026935s) [noise=true , address=77.90.31.13:4632 , piece_id=UPXZMVVQKHVEKRLOQOUI3NRFVPZ6EVV3GPPMZAINHATAFRTNOBUA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:13iPed9zYrthvh36xo3k4gsqm4tz86iVkNvAr3wauuArQLquyS, address:77.90.31.13:4632): protocol: rpc: dial tcp 77.90.31.13:4632: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399592421 (3h17m15.334046562s) [noise=true , address=77.90.31.14:4669 , piece_id=U5I622KDIZ7HDTQUJM7CMDS6DJ3LPDJKWYZKVVHDQTY3Z6URWXYA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:12wgv8uy7FaviWBw3H8ZS2NfJNnZ5F5PYyxxJrYaJsfGVv55Q8T, address:77.90.31.14:4669): protocol: rpc: dial tcp 77.90.31.14:4669: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399635924 (3h17m15.334090042s) [noise=true , address=77.90.31.12:4624 , piece_id=5QYKED24FNXRXY3M5K5FLKOC6GN3GKYTOS3NCWP36QXPNYEAAENA, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1KxpgfHEQjqnumtfhQ2TEYCgZPFbUQVShfJA7dgNbnZXpncMnV, address:77.90.31.12:4624): protocol: rpc: dial tcp 77.90.31.12:4624: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399794086 (3h17m15.334248222s) [noise=true , address=77.90.31.11:4581 , piece_id=J2Z6JEPECHBXOHQRLSORGZFLMFAHKXAXH2RHZV44B3EASHJRXZ7A, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1MP1x3VuzX7cQPmD9pY3Cw1v6W2Q5LqUDdKesqX31vL9JLTRAp, address:77.90.31.11:4581): protocol: write tcp 139.59.144.22:54212->77.90.31.11:4581: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399835370 (3h17m15.334289512s) [noise=true , address=77.90.31.10:4532 , piece_id=MALUR6KD73HNUAS7RBXKFA2QMDIKA7NQBGFNVYMEX7G53LMY2WMQ, seg_pos={0 0}, upload=2852]: Done uploading piece. err: ecclient: upload failed (node:1T6kkKvTNR5MZZijjr1vMU6g75YkoRpzMvGSgQCqRnfCm9JULe, address:77.90.31.10:4532): protocol: rpc: dial tcp 77.90.31.10:4532: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:34:10.399875448 (3h17m15.334329567s) [seg_pos={0 0}, upload=2852]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:34:10.497548320 (3h17m15.432002443s) [upload=2852]: Done uploading segments...
Feb 17 23:34:10.497582479 (3h17m15.432036614s) [upload=2852]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1m0.100484437s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [...............XXXX.........................................] duration:
1.60321044s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:33:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.603234786s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...................XX.......................................] duration:
2.813359707s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:33:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.813384687s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
25 out of 60 is failed (threshold=3) [.............................XX.XX...X.XXXXXXXXXXXXXXXXX.XXX] duration:
1m0.110591229s trace:
https://jaeger.dev.storj.io/trace/296bb197cb3e3cf3
FAIL (2026-02-17 23:33:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
434512478s) [seg_pos={0 0}, upload=2838]: Begin upload segment...
Feb 17 23:32:10.500126165 (3h15m15.434580299s) [seg_pos={0 0}, upload=2838]: Done begin upload segment.
Feb 17 23:32:10.500163705 (3h15m15.434617849s) [noise=true , address=45.91.116.4:4946 , piece_id=HGQ7AB44YKTCAUK4K7KBP35WV7DLGL4XWLM32PF7OWRJZCUCZIZA, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.500184361 (3h15m15.4346385s) [noise=true , address=145.79.227.2:4123 , piece_id=ZR4SVAVHGRBI75EGOTER57QFBQGSV3F3Z5IY5NDYSQU2PIFHWZRA, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.500862438 (3h15m15.435316574s) [noise=true , address=77.90.31.18:4820 , piece_id=DEDQVPLSCWNLBUCUEO6G6S4JS2RVV7X76WJIM5LPTOO55R2W2FLA, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.501326990 (3h15m15.435781133s) [noise=true , address=193.149.164.66:4969 , piece_id=JW2OXFKF4SLRDUGFSLGGTZJQ4D3SS7I44KGSWM43ISF2MIZSQLPA, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.501619254 (3h15m15.436073391s) [noise=true , address=77.90.31.21:4895 , piece_id=U7D6HE4YTQEBOXROI4TNQ6SDVZCHX72BSQNNJ2SEDWD2KO2H4GWQ, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.502102796 (3h15m15.436556938s) [noise=true , address=77.90.31.11:4579 , piece_id=RTLEMEMEFWXSEYVTOBPNHTICFOOQR6IFDJLWPYNZUJOXFTEC6M3Q, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.502623004 (3h15m15.437077129s) [noise=true , address=193.149.164.67:4008 , piece_id=UOKSNNCHT6SY5X47FSB7JDTKXU7C7HOW6CMN5HBJBSMKR4OSXCEQ, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.502827758 (3h15m15.437281901s) [noise=true , address=45.91.116.9:4289 , piece_id=RJVGYF65GPGM3O3IN3VUEBHBMU3NXBD6SJMZMPGY3VPVSB2D5XZQ, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.503304333 (3h15m15.437758478s) [noise=true , address=64.49.13.139:4476 , piece_id=K27UHKD464H2WEWC732IXJ2MUJ3YP75BLS7LXFZZPXYJ3HYW3ERA, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.503773754 (3h15m15.438227894s) [noise=true , address=77.90.31.19:4872 , piece_id=VOCH47DSYKA2WF3FAZPNB7I2IRE4NUZH2UY5Q5M5MASMEOL2M7EQ, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.504215786 (3h15m15.438669925s) [noise=true , address=77.90.31.16:4749 , piece_id=6WO7LJU6WZD6KWXVO73HQC3YQTK3WCDJEXLOSR5PRJBXO2NUIHHQ, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.504821366 (3h15m15.439275508s) [noise=true , address=77.90.31.14:4677 , piece_id=DQB2WCFP26R42KMT7XLKZGNLT26V42GLNE4EOLGXWRFTZTD3LWQQ, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.505383049 (3h15m15.439837189s) [noise=true , address=5.231.73.3:4031 , piece_id=Y4RJBUWPO35C7HZLYWUMWTLJB5PSIHGIAGI4JXDPYYD6FSSPXZWA, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.506003469 (3h15m15.440457665s) [noise=true , address=145.79.227.21:4492 , piece_id=JTLXJDAWFWCV4JAUTXW6DYQSDLZSGIQN5BPNJDMEZ5PE2IQ4ABBQ, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.506631488 (3h15m15.441085636s) [noise=true , address=193.149.164.68:4051 , piece_id=YNWX4PJXNZ6BINS7FZHWKV4XVNUN3DFFXWPVBYWT4TOWT6GMTO2A, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.507239148 (3h15m15.44169329s) [noise=true , address=193.149.164.69:4082 , piece_id=FQYLHGJBN6N322RHVKETBJCFAFO3L2JUHTSCIJIUMTGEXDMVNQSA, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.507882882 (3h15m15.442337023s) [noise=true , address=77.90.31.12:4622 , piece_id=ES5HZQGJOYSBI3MND5JWLILM7ZD7T76KZPSZJQVLX4RG2AARWJ4A, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:10.509522926 (3h15m15.443977078s) [noise=true , address=145.79.227.44:4227 , piece_id=CV7LHSADWHED4LXV43CARVCCBSUXYGWGNACRXL237V37D7RBDM7A, seg_pos={0 0}, upload=2838]: Uploading piece...
Feb 17 23:32:11.060280048 (3h15m15.994734183s) [upload=2838]: Next returned nil segment
Feb 17 23:32:11.060299680 (3h15m15.994753823s) [upload=2838]: Waiting for error group managing segments...
Feb 17 23:32:11.206715889 (3h15m16.14117003s) [noise=true , address=145.79.227.2:4123 , piece_id=ZR4SVAVHGRBI75EGOTER57QFBQGSV3F3Z5IY5NDYSQU2PIFHWZRA, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.215499557 (3h15m16.149953693s) [noise=true , address=45.91.116.4:4946 , piece_id=HGQ7AB44YKTCAUK4K7KBP35WV7DLGL4XWLM32PF7OWRJZCUCZIZA, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.221320505 (3h15m16.155774661s) [noise=true , address=45.91.116.9:4289 , piece_id=RJVGYF65GPGM3O3IN3VUEBHBMU3NXBD6SJMZMPGY3VPVSB2D5XZQ, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.223758421 (3h15m16.158212553s) [noise=true , address=145.79.227.21:4492 , piece_id=JTLXJDAWFWCV4JAUTXW6DYQSDLZSGIQN5BPNJDMEZ5PE2IQ4ABBQ, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.427939048 (3h15m16.362393189s) [noise=true , address=5.231.73.3:4031 , piece_id=Y4RJBUWPO35C7HZLYWUMWTLJB5PSIHGIAGI4JXDPYYD6FSSPXZWA, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.722369563 (3h15m16.656823701s) [noise=true , address=193.149.164.67:4008 , piece_id=UOKSNNCHT6SY5X47FSB7JDTKXU7C7HOW6CMN5HBJBSMKR4OSXCEQ, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.741063001 (3h15m16.675517138s) [noise=true , address=193.149.164.69:4082 , piece_id=FQYLHGJBN6N322RHVKETBJCFAFO3L2JUHTSCIJIUMTGEXDMVNQSA, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.742571169 (3h15m16.677025302s) [noise=true , address=64.49.13.139:4476 , piece_id=K27UHKD464H2WEWC732IXJ2MUJ3YP75BLS7LXFZZPXYJ3HYW3ERA, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.748148414 (3h15m16.682602552s) [noise=true , address=193.149.164.68:4051 , piece_id=YNWX4PJXNZ6BINS7FZHWKV4XVNUN3DFFXWPVBYWT4TOWT6GMTO2A, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.753012955 (3h15m16.687467089s) [noise=true , address=193.149.164.66:4969 , piece_id=JW2OXFKF4SLRDUGFSLGGTZJQ4D3SS7I44KGSWM43ISF2MIZSQLPA, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:32:11.787948854 (3h15m16.722402995s) [noise=true , address=145.79.227.44:4227 , piece_id=CV7LHSADWHED4LXV43CARVCCBSUXYGWGNACRXL237V37D7RBDM7A, seg_pos={0 0}, upload=2838]: Done uploading piece. err: <nil>
Feb 17 23:33:10.116978708 (3h16m15.051432849s) [noise=true , address=77.90.31.16:4749 , piece_id=6WO7LJU6WZD6KWXVO73HQC3YQTK3WCDJEXLOSR5PRJBXO2NUIHHQ, seg_pos={0 0}, upload=2838]: Done uploading piece. err: ecclient: upload failed (node:1EWSdFD1GDpqBHwXHZiDzR9nj8wE4vzmwnMbTrXrveiQXkUVC7, address:77.90.31.16:4749): protocol: rpc: dial tcp 77.90.31.16:4749: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:33:10.117127840 (3h16m15.051581975s) [noise=true , address=77.90.31.12:4622 , piece_id=ES5HZQGJOYSBI3MND5JWLILM7ZD7T76KZPSZJQVLX4RG2AARWJ4A, seg_pos={0 0}, upload=2838]: Done uploading piece. err: ecclient: upload failed (node:12MQryCwJkVN1JhFm4QqhypX3WoMDR9rdjELj4oYoVbpa6U823Z, address:77.90.31.12:4622): protocol: write tcp 139.59.144.22:55226->77.90.31.12:4622: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:33:10.117232431 (3h16m15.051686588s) [noise=true , address=77.90.31.14:4677 , piece_id=DQB2WCFP26R42KMT7XLKZGNLT26V42GLNE4EOLGXWRFTZTD3LWQQ, seg_pos={0 0}, upload=2838]: Done uploading piece. err: ecclient: upload failed (node:1afoGj5ZD9twztzCN957MdgkWwydWT3pkGkXXtStw3P6hZBoqK, address:77.90.31.14:4677): protocol: rpc: dial tcp 77.90.31.14:4677: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:33:10.117408862 (3h16m15.051863005s) [noise=true , address=77.90.31.18:4820 , piece_id=DEDQVPLSCWNLBUCUEO6G6S4JS2RVV7X76WJIM5LPTOO55R2W2FLA, seg_pos={0 0}, upload=2838]: Done uploading piece. err: ecclient: upload failed (node:12bFT8wMeMrsm5byCeSjJQQUXCWmDtyBqiiu1KqpXMoxTHAakV9, address:77.90.31.18:4820): protocol: rpc: dial tcp 77.90.31.18:4820: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:33:10.117657086 (3h16m15.052111225s) [noise=true , address=77.90.31.11:4579 , piece_id=RTLEMEMEFWXSEYVTOBPNHTICFOOQR6IFDJLWPYNZUJOXFTEC6M3Q, seg_pos={0 0}, upload=2838]: Done uploading piece. err: ecclient: upload failed (node:12dRcuBLuT2ybWGzdkp4vyo7e37qrZk6SBpWWHty5WKRoVHp1p, address:77.90.31.11:4579): protocol: write tcp 139.59.144.22:46796->77.90.31.11:4579: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:33:10.117816949 (3h16m15.052271084s) [noise=true , address=77.90.31.19:4872 , piece_id=VOCH47DSYKA2WF3FAZPNB7I2IRE4NUZH2UY5Q5M5MASMEOL2M7EQ, seg_pos={0 0}, upload=2838]: Done uploading piece. err: ecclient: upload failed (node:127ckxo8rSfJjtVZHEquzgsHGG78DiyoSRdqykfrzbfw1n5tfWL, address:77.90.31.19:4872): protocol: write tcp 139.59.144.22:45846->77.90.31.19:4872: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:33:10.118035686 (3h16m15.052489835s) [noise=true , address=77.90.31.21:4895 , piece_id=U7D6HE4YTQEBOXROI4TNQ6SDVZCHX72BSQNNJ2SEDWD2KO2H4GWQ, seg_pos={0 0}, upload=2838]: Done uploading piece. err: ecclient: upload failed (node:12S7WBBQ1Nq8jbeGrZiRKTk23CdJC31hPW2vU1aaM5SsHeVyeRH, address:77.90.31.21:4895): protocol: write tcp 139.59.144.22:43094->77.90.31.21:4895: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:33:10.118074857 (3h16m15.05252899s) [seg_pos={0 0}, upload=2838]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Feb 17 23:33:10.224646337 (3h16m15.15910046s) [upload=2838]: Done uploading segments...
Feb 17 23:33:10.224681934 (3h16m15.159136057s) [upload=2838]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 1m0.110615492s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....................XX......................................] duration:
2.896699691s trace:
https://jaeger.dev.storj.io/trace/478c1cba3de9cc23
FAIL (2026-02-17 23:32:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.896721274s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [................XXXX........................................] duration:
2.147627357s trace:
https://jaeger.dev.storj.io/trace/4f259542fd8bf4d2
FAIL (2026-02-17 23:32:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.147650837s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
24 out of 60 is failed (threshold=3) [..............................XX.XX...X.XXXXXXXXXXXXXXXXX.XX] duration:
1m0.102719774s trace:
https://jaeger.dev.storj.io/trace/62c08cf38f83d54a
FAIL (2026-02-17 23:31:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
Feb 17 23:30:59.745996307 (3h14m4.680450449s) [upload=2819]: Starting upload
Feb 17 23:30:59.746140966 (3h14m4.680595104s) [upload=2819]: Done starting upload
Feb 17 23:30:59.746269264 (3h14m4.680723386s) [upload=2819]: Uploading segments...
Feb 17 23:30:59.746628909 (3h14m4.681083052s) [upload=2819]: Got next segment. Inline: false
Feb 17 23:30:59.746720255 (3h14m4.681174394s) [upload=2819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 17 23:30:59.746727265 (3h14m4.681181405s) [upload=2819]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 17 23:30:59.854408184 (3h14m4.788862319s) [seg_pos={0 0}, upload=2819]: Begin upload segment...
Feb 17 23:30:59.854481047 (3h14m4.78893517s) [seg_pos={0 0}, upload=2819]: Done begin upload segment.
Feb 17 23:30:59.854515340 (3h14m4.788969475s) [noise=true , address=77.90.31.21:4888 , piece_id=WN5UC2EIS5545XWG33T3N3HAJCSRXOMFT7ENO7SDHL3GHZS6GNFQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.854929385 (3h14m4.789383505s) [noise=true , address=193.149.164.69:4080 , piece_id=NJMG6QES35ONCTZCKRRQAOLQQTCHDX6NSA5SMXYXBXKKIOGC2VUA, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.855304709 (3h14m4.78975883s) [noise=true , address=77.90.31.11:4576 , piece_id=DGQZNEJSSYTKANWSMIWSHND4OFYBP5O7ZAZC25CZD6RABLTXUCPA, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.855818835 (3h14m4.790272977s) [noise=true , address=193.149.164.66:4988 , piece_id=ONGP7ENYBX3HN3O3FVMLSH6WYAMKGQCDHBS3RUKUFDCXP3TFAKZQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.856352869 (3h14m4.790807011s) [noise=true , address=45.91.116.8:4061 , piece_id=GMFTSWUKV7WQZ4YATKZQ3MGJOHAGF4TJI3TJJXLKWOSDKYQXYBJA, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.856778960 (3h14m4.791233106s) [noise=true , address=145.79.227.19:4088 , piece_id=DQJ7QMAE5QBE74REAOZUWA3OC2HLBQE7VDDRFNRNQB5MRN7OVBEQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.857010789 (3h14m4.791464936s) [noise=true , address=145.79.227.32:4854 , piece_id=VVVZ4XCSNPWKGNJRRTRAO2OL33Z6MSXLOLRGENGW4QQQSG7ZBTDQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.857378734 (3h14m4.791832881s) [noise=true , address=77.90.31.19:4845 , piece_id=LOOSS3Q63TYEGGG5PZPDBWH3YSL2BDAXFICTBYH5BN3Q2JPAF2EQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.857513855 (3h14m4.791968001s) [noise=true , address=193.149.164.70:4118 , piece_id=JQE4UR7226YJSLELIO4ZE66IXKHEQ5GPBK4EN7NLVTTETZYQS6SA, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.857903531 (3h14m4.792357668s) [noise=true , address=145.79.227.17:4990 , piece_id=NDE25XOKZMJN4U7EJPE7GM4NXRP54MXEYIAPGDFOBO7JM4ZRQZEQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.857985898 (3h14m4.792440033s) [noise=true , address=45.91.116.13:4175 , piece_id=5DMFQJ7AQZNNNQVZ433SFM6WLAVASU35UQBTMTF7F56XQ2NEXUZQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.858436357 (3h14m4.79289049s) [noise=true , address=145.79.227.39:4041 , piece_id=LUTTN4JRS6QPWXZDHKZ66UACVX54OROJBFI4BT6GEJUZ2Y5JCG5Q, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.858524677 (3h14m4.792978821s) [noise=true , address=77.90.31.16:4737 , piece_id=3PLUZHIIPO45MLXXKAX4EXEFBMQBSCEECIZOLX7J5QZQXIJZ35MQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.858880329 (3h14m4.793334473s) [noise=true , address=193.149.164.68:4046 , piece_id=XVIIVNFIAWC6ID6RYPNC3ADVG4YND6QRL2UI4CP6BLQ54KL6O7BA, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.859100936 (3h14m4.793555078s) [noise=true , address=145.79.227.41:4109 , piece_id=ZXKD7CKLCYE6C6MFJKOVYUVCGTPJL3T2UIEUQVCYKPPZNVACWYWA, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.859625578 (3h14m4.794079704s) [noise=true , address=77.90.31.13:4648 , piece_id=JIKISDICSDOISFGZLTCQLESUHJ4I37J6JRUKUCF4F3FQTMIQLDYA, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.860047029 (3h14m4.794501153s) [noise=true , address=77.90.31.15:4728 , piece_id=DQ4WODZFI26YX6TQD2IDONOKAEJWCMAVVQEAJUNAWYQHKE4H6NSQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:30:59.860439949 (3h14m4.794894091s) [noise=true , address=45.91.116.10:4103 , piece_id=NNK65PFWJEBCDAELBNCIURJXRPSR4OELHFNDS4TA6HUC6LI2K7KQ, seg_pos={0 0}, upload=2819]: Uploading piece...
Feb 17 23:31:00.419952746 (3h14m5.354406865s) [upload=2819]: Next returned nil segment
Feb 17 23:31:00.419972925 (3h14m5.354427072s) [upload=2819]: Waiting for error group managing segments...
Feb 17 23:31:00.571442003 (3h14m5.505896141s) [noise=true , address=45.91.116.13:4175 , piece_id=5DMFQJ7AQZNNNQVZ433SFM6WLAVASU35UQBTMTF7F56XQ2NEXUZQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:00.584900795 (3h14m5.51935494s) [noise=true , address=145.79.227.39:4041 , piece_id=LUTTN4JRS6QPWXZDHKZ66UACVX54OROJBFI4BT6GEJUZ2Y5JCG5Q, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:00.588321782 (3h14m5.52277593s) [noise=true , address=145.79.227.19:4088 , piece_id=DQJ7QMAE5QBE74REAOZUWA3OC2HLBQE7VDDRFNRNQB5MRN7OVBEQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:00.592715890 (3h14m5.52717005s) [noise=true , address=45.91.116.8:4061 , piece_id=GMFTSWUKV7WQZ4YATKZQ3MGJOHAGF4TJI3TJJXLKWOSDKYQXYBJA, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:00.602326097 (3h14m5.536780238s) [noise=true , address=145.79.227.17:4990 , piece_id=NDE25XOKZMJN4U7EJPE7GM4NXRP54MXEYIAPGDFOBO7JM4ZRQZEQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:00.606154757 (3h14m5.540608907s) [noise=true , address=145.79.227.32:4854 , piece_id=VVVZ4XCSNPWKGNJRRTRAO2OL33Z6MSXLOLRGENGW4QQQSG7ZBTDQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:00.634372782 (3h14m5.568826926s) [noise=true , address=145.79.227.41:4109 , piece_id=ZXKD7CKLCYE6C6MFJKOVYUVCGTPJL3T2UIEUQVCYKPPZNVACWYWA, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:00.643521458 (3h14m5.577975593s) [noise=true , address=45.91.116.10:4103 , piece_id=NNK65PFWJEBCDAELBNCIURJXRPSR4OELHFNDS4TA6HUC6LI2K7KQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:01.092839669 (3h14m6.027293807s) [noise=true , address=193.149.164.70:4118 , piece_id=JQE4UR7226YJSLELIO4ZE66IXKHEQ5GPBK4EN7NLVTTETZYQS6SA, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:01.103379674 (3h14m6.037833806s) [noise=true , address=193.149.164.69:4080 , piece_id=NJMG6QES35ONCTZCKRRQAOLQQTCHDX6NSA5SMXYXBXKKIOGC2VUA, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:01.113087706 (3h14m6.047541845s) [noise=true , address=193.149.164.66:4988 , piece_id=ONGP7ENYBX3HN3O3FVMLSH6WYAMKGQCDHBS3RUKUFDCXP3TFAKZQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:01.119390032 (3h14m6.053844165s) [noise=true , address=193.149.164.68:4046 , piece_id=XVIIVNFIAWC6ID6RYPNC3ADVG4YND6QRL2UI4CP6BLQ54KL6O7BA, seg_pos={0 0}, upload=2819]: Done uploading piece. err: <nil>
Feb 17 23:31:59.476616473 (3h15m4.411070613s) [noise=true , address=77.90.31.21:4888 , piece_id=WN5UC2EIS5545XWG33T3N3HAJCSRXOMFT7ENO7SDHL3GHZS6GNFQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: ecclient: upload failed (node:12BiqwcDuvobZabx3jxDhXf27Ei8oLFS6wLp3qbpZCtJ4ZhQEZA, address:77.90.31.21:4888): protocol: rpc: dial tcp 77.90.31.21:4888: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 17 23:31:59.476938293 (3h15m4.41139243s) [noise=true , address=77.90.31.11:4576 , piece_id=DGQZNEJSSYTKANWSMIWSHND4OFYBP5O7ZAZC25CZD6RABLTXUCPA, seg_pos={0 0}, upload=2819]: Done uploading piece. err: ecclient: upload failed (node:1br8Q73A3rDz8b6fZxop71zh5vdEcdye9D7ta8aBpsctwLBfAv, address:77.90.31.11:4576): protocol: rpc: dial tcp 77.90.31.11:4576: i/o timeout; context deadline exceeded
Feb 17 23:31:59.477207616 (3h15m4.411661759s) [noise=true , address=77.90.31.19:4845 , piece_id=LOOSS3Q63TYEGGG5PZPDBWH3YSL2BDAXFICTBYH5BN3Q2JPAF2EQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: ecclient: upload failed (node:1ysgMdZQcnmzQ5Vfv3RadZN8widTZKrqDnDSthxkYEUXKgRxxF, address:77.90.31.19:4845): protocol: context deadline exceeded; context deadline exceeded
Feb 17 23:31:59.477437220 (3h15m4.411891362s) [noise=true , address=77.90.31.15:4728 , piece_id=DQ4WODZFI26YX6TQD2IDONOKAEJWCMAVVQEAJUNAWYQHKE4H6NSQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: ecclient: upload failed (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo, address:77.90.31.15:4728): protocol: context deadline exceeded; context deadline exceeded
Feb 17 23:31:59.477573711 (3h15m4.412027833s) [noise=true , address=77.90.31.16:4737 , piece_id=3PLUZHIIPO45MLXXKAX4EXEFBMQBSCEECIZOLX7J5QZQXIJZ35MQ, seg_pos={0 0}, upload=2819]: Done uploading piece. err: ecclient: upload failed (node:12pAVncc8nskMA4guej8rBKjXSY6qrCScMbq8edx6dpWk4vf8rM, address:77.90.31.16:4737): protocol: context deadline exceeded; context deadline exceeded
Feb 17 23:31:59.477672715 (3h15m4.412126857s) [noise=true , address=77.90.31.13:4648 , piece_id=JIKISDICSDOISFGZLTCQLESUHJ4I37J6JRUKUCF4F3FQTMIQLDYA, seg_pos={0 0}, upload=2819]: Done uploading piece. err: ecclient: upload failed (node:1niTsTaHjA6LoMMrkEsSnLcRDhZtVAjxsbXUXVeXzhUp3FPbWh, address:77.90.31.13:4648): protocol: context deadline exceeded; context deadline exceeded
Feb 17 23:31:59.477715264 (3h15m4.412169383s) [seg_pos={0 0}, upload=2819]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Feb 17 23:31:59.576649364 (3h15m4.511103497s) [upload=2819]: Done uploading segments...
Feb 17 23:31:59.576691014 (3h15m4.51114515s) [upload=2819]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1m0.102751391s
core
uplink
us-select-1
us1-placement-20
|