Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 848.526223ms FAIL (2025-08-09 12:11:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 848.55718ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.427825063s FAIL (2025-08-09 12:11:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.427853957s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.269695828s FAIL (2025-08-09 12:11:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.269722565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 845.487668ms FAIL (2025-08-09 12:10:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 845.520704ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.485198326s FAIL (2025-08-09 12:10:51 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.485240812s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.300288665s FAIL (2025-08-09 12:10:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.300317703s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 914.73803ms FAIL (2025-08-09 12:10:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 914.770234ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.544023082s FAIL (2025-08-09 12:09:51 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.544062401s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 897.113049ms FAIL (2025-08-09 12:09:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 897.154414ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.314194586s FAIL (2025-08-09 12:09:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.31422247s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.244480819s FAIL (2025-08-09 12:09:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.244498919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 845.176174ms FAIL (2025-08-09 12:09:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 845.218835ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.402047844s FAIL (2025-08-09 12:08:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.402079595s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.315391694s FAIL (2025-08-09 12:08:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.315424775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.392993049s FAIL (2025-08-09 12:08:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.393025166s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 827.036167ms FAIL (2025-08-09 12:07:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 827.072549ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.314832845s FAIL (2025-08-09 12:07:50 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.314861225s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 833.707071ms FAIL (2025-08-09 12:07:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 833.741201ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.306488076s FAIL (2025-08-09 12:07:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.306516746s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 888.426792ms FAIL (2025-08-09 12:07:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 888.454861ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.318822992s FAIL (2025-08-09 12:06:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.318850793s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.37084369s FAIL (2025-08-09 12:06:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.370870765s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.447263798s FAIL (2025-08-09 12:06:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.447302944s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 857.593754ms FAIL (2025-08-09 12:06:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 857.628449ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 3.930782441s FAIL (2025-08-09 12:05:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 3.930818209s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.276863679s FAIL (2025-08-09 12:05:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.276899728s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.27274856s FAIL (2025-08-09 12:05:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.27279621s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.325786002s FAIL (2025-08-09 12:04:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.325819017s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 954.45797ms FAIL (2025-08-09 12:04:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 954.489356ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.58089553s FAIL (2025-08-09 12:04:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.580925318s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.376226075s FAIL (2025-08-09 12:04:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.376261386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 905.962854ms FAIL (2025-08-09 12:04:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 905.991716ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.272438214s FAIL (2025-08-09 12:03:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.272470985s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.416015952s FAIL (2025-08-09 12:03:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.416105794s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.368126476s FAIL (2025-08-09 12:03:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.368161487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.160970654s FAIL (2025-08-09 12:03:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 1.161017267s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.664956407s FAIL (2025-08-09 12:02:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.664988305s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.143307777s FAIL (2025-08-09 12:02:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 1.14334359s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.301626886s FAIL (2025-08-09 12:02:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.301657312s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.561613809s FAIL (2025-08-09 12:02:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.561801483s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.347075694s FAIL (2025-08-09 12:01:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.347100307s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 893.980606ms FAIL (2025-08-09 12:01:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 894.01358ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.356135245s FAIL (2025-08-09 12:01:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.356161105s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.465584309s FAIL (2025-08-09 12:01:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.465655379s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.34991831s FAIL (2025-08-09 12:00:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.349947932s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 983.283257ms FAIL (2025-08-09 12:00:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 983.31134ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.540351133s FAIL (2025-08-09 12:00:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 4226018 (18h11m29.273323277s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314415633 (18h11m29.273512881s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.314726594 (18h11m29.273823839s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315035831 (18h11m29.274133076s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315361473 (18h11m29.274458723s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.315801774 (18h11m29.274899021s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316102423 (18h11m29.27519967s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316378502 (18h11m29.275475752s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.316709100 (18h11m29.275806353s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317222442 (18h11m29.276319697s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.317598381 (18h11m29.276695629s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.318004669 (18h11m29.277101918s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Uploading piece... Aug 9 11:37:07.949397504 (18h11m29.90849476s) [upload=18948]: Next returned nil segment Aug 9 11:37:07.949420624 (18h11m29.908517904s) [upload=18948]: Waiting for error group managing segments... Aug 9 11:37:08.089551930 (18h11m30.048649175s) [noise=true , address=104.36.82.6:4874 , piece_id=Z6LRH3WR3QWODHLC2OP3JOQXABIIFASZUK3SS7N33KPN2OKQXDPA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.092412944 (18h11m30.051510189s) [noise=true , address=104.36.82.12:4149 , piece_id=BKU2JMJSDHBTU345CVZKYTGOONXPP2PG6HBY2AQI2LTQKQKE6U6Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.099385074 (18h11m30.058482323s) [noise=true , address=104.36.82.14:4223 , piece_id=6H22IPHTTDZ2QPKBWZ5E3LLXL4EQTBHA6P47MXMVNG4SH2Q3EUTA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:08.180090657 (18h11m30.139187895s) [noise=true , address=104.36.82.8:4972 , piece_id=74M6IKUJ4P43BJ3SIGWJRU346DEQ3D6QI4B2QUM2QKXWWY5YS6CQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:24.004796612 (18h11m45.963893849s) [noise=true , address=45.91.116.6:4974 , piece_id=KX5HPZP2RD7QP3MVHEO53GOGFQZJHWWXX5OWPMQPCLS4KD2RMHNQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:34.496190916 (18h11m56.455288156s) [noise=true , address=45.91.116.11:4117 , piece_id=L2UE4CFNKWOCXU24CCXOPONUYV25RX5Z7MDLLRAX6S67PXTVL6KQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: <nil> Aug 9 11:37:36.097756932 (18h11m58.056854182s) [noise=true , address=45.91.116.4:4939 , piece_id=LHJ23G5MQRYBRCQTYZK7CM27GTOD2RLZFJYE57Z7FBF44IR3HQ6A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.097973210 (18h11m58.057070448s) [noise=true , address=45.91.116.5:4299 , piece_id=AWBWJMYSVFAXP4XR75L3NHOO2UWHIUQFI2PNB5LJMJBGGLS2H22Q, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: expected piece hash; context deadline exceeded; EOF Aug 9 11:37:36.098161794 (18h11m58.05725904s) [noise=true , address=45.91.116.5:4297 , piece_id=QDDYI7DWOCVI67E2L745ECAFNJJGFLSDYHW2JHQOL4GW7NAI5BYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46622->45.91.116.5:4297: use of closed network connection Aug 9 11:37:36.098207799 (18h11m58.057305037s) [noise=true , address=45.91.116.8:4048 , piece_id=IHQMY2D5JLMBKMAGZELQBRCT2J5ILZRF7ERW2KLFKGFBUUIFXNKA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50804->45.91.116.8:4048: use of closed network connection Aug 9 11:37:36.098245218 (18h11m58.057342454s) [noise=true , address=45.91.116.14:4190 , piece_id=52ZKME7PNDI3RCSWSZQKYTBICCKQS7RLMHJVPGYGV44GLTVCAQ3A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32826->45.91.116.14:4190: use of closed network connection Aug 9 11:37:36.098287535 (18h11m58.057384779s) [noise=true , address=45.91.116.12:4226 , piece_id=KYCTUVJZH2QWVDFE35UZWPSMZ2ZZ2JKQDJCKEM7VEZN6OWEWVTZQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55374->45.91.116.12:4226: use of closed network connection Aug 9 11:37:36.098318065 (18h11m58.057415301s) [noise=true , address=77.90.31.19:4842 , piece_id=DSBB2FNGQL76Y5FFG5BI4ZWZL7XRR5NJS263F5E5KN4BBIXZA75A, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49024->77.90.31.19:4842: use of closed network connection Aug 9 11:37:36.098362497 (18h11m58.057459737s) [noise=true , address=77.90.31.11:4553 , piece_id=IMSNXEECWHIASEQYZRASRJMT2PJYXDJ65OFBIEFUABVYLIXCKUYQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58474->77.90.31.11:4553: use of closed network connection Aug 9 11:37:36.098370733 (18h11m58.057467969s) [noise=true , address=77.90.31.14:4662 , piece_id=KE2HYELFSASWLF2QUTIGUQW7JQMBW7LNHEA3R5NPUKIV4VGUSXBA, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50830->77.90.31.14:4662: use of closed network connection Aug 9 11:37:36.098415055 (18h11m58.057512295s) [noise=true , address=45.91.116.12:4225 , piece_id=7TIJNRDL75TBLIMWKWOK7WPTIQIJ7XCH6R6YBDG45NET7UW6G6LQ, seg_pos={0 0}, upload=18948]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40390->45.91.116.12:4225: use of closed network connection Aug 9 11:37:36.098439035 (18h11m58.057536289s) [seg_pos={0 0}, upload=18948]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 9 11:37:36.327886327 (18h11m58.286983577s) [upload=18948]: Done uploading segments... Aug 9 11:37:36.327920391 (18h11m58.287017643s) [upload=18948]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 2.540425586s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.382824003s FAIL (2025-08-09 11:59:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 9 11:31:24.050233439 (18h5m46.009330686s) [upload=18850]: Starting upload Aug 9 11:31:24.050296503 (18h5m46.009393749s) [upload=18850]: Done starting upload Aug 9 11:31:24.050390468 (18h5m46.009487714s) [upload=18850]: Uploading segments... Aug 9 11:31:24.050779018 (18h5m46.009876267s) [upload=18850]: Got next segment. Inline: false Aug 9 11:31:24.050811109 (18h5m46.009908354s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:31:24.050816322 (18h5m46.00991357s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 11:31:24.257548098 (18h5m46.216645343s) [seg_pos={0 0}, upload=18850]: Begin upload segment... Aug 9 11:31:24.257604682 (18h5m46.216701917s) [seg_pos={0 0}, upload=18850]: Done begin upload segment. Aug 9 11:31:24.257630905 (18h5m46.216728153s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.257646946 (18h5m46.2167442s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258091247 (18h5m46.217188502s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258475545 (18h5m46.217572796s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258638130 (18h5m46.217735377s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258789065 (18h5m46.21788631s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.258992397 (18h5m46.218089643s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259127956 (18h5m46.218225212s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259586700 (18h5m46.218683948s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.259892072 (18h5m46.218989319s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260172760 (18h5m46.219270008s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260484239 (18h5m46.219581484s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260599488 (18h5m46.219696746s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.260922775 (18h5m46.22002002s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261202742 (18h5m46.22029999s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261523530 (18h5m46.220620785s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.261609933 (18h5m46.220707182s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.262571015 (18h5m46.22166826s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Uploading piece... Aug 9 11:31:24.823957412 (18h5m46.783054657s) [upload=18850]: Next returned nil segment Aug 9 11:31:24.823997893 (18h5m46.783095143s) [upload=18850]: Waiting for error group managing segments... Aug 9 11:31:24.973869616 (18h5m46.932966849s) [noise=true , address=45.91.116.9:4275 , piece_id=ZWTSMDLYMCXUDSX2OZD7HX2GGZ37R4GUU2SMKP3F3IMZOUTCDASQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.976820739 (18h5m46.935917988s) [noise=true , address=104.36.82.9:4018 , piece_id=KYJ2VLOYNRRVGLV7YXDHR5CBXXDFN5XOXO7RLYCACZILF5NMF7RA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.977052015 (18h5m46.936149255s) [noise=true , address=45.91.116.14:4221 , piece_id=RSJHTUJI37JS22GLRQVZYJ64J2TSL4NS7PNGEA53TMOJKO6HAELQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.983920372 (18h5m46.943017618s) [noise=true , address=104.36.82.2:4761 , piece_id=YIBDR7Y2VGWEUTQ2446K2ABNQPOVPK2S5MUFOMOZHXRHU7DJ7L7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984021492 (18h5m46.94311873s) [noise=true , address=104.36.82.59:4326 , piece_id=U7LQCUN3JWLYJLTVU3G23KR7N2XK5JWW7LWBTU2M64F6PTV7YWUQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.984755330 (18h5m46.943852577s) [noise=true , address=104.36.82.36:4630 , piece_id=QRVWQZEFIIYPPIQIOS32BCGCBCPNTYBL24MBVEGH56U67QMX7JHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:24.990369703 (18h5m46.94946694s) [noise=true , address=104.36.82.45:4924 , piece_id=Z6RL4TETD5EY4XXPVKMNSXLJPLCWFJJYWVF6CFF7YJLW3GLUREBA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004636078 (18h5m46.963733324s) [noise=true , address=104.36.82.9:4011 , piece_id=AQNQGGCT3MZEE7TRKMTZDT3HNNYZ5WUSERK3YQVYOC2LZCUKXVHQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.004692950 (18h5m46.963790197s) [noise=true , address=104.36.82.61:4387 , piece_id=SJ5C34G35IUMNHBEDRXUYPCJQ5NASH6CA7IP337LET3URPL4AI7Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.005135487 (18h5m46.964232731s) [noise=true , address=104.36.82.8:4001 , piece_id=GALLKA5LMCVRXMO2U4KWE2I2ZAP6LFBKV64UJXYUE2DID4VD7RHA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008245020 (18h5m46.967342269s) [noise=true , address=104.36.82.22:4515 , piece_id=FX7SCOWUHQAXGCTNGE32BQOJAMLRP2RLW6IN3ZXT37DGG7UCL67Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.008631208 (18h5m46.967728458s) [noise=true , address=104.36.82.18:4387 , piece_id=XRWROTRDATNM7QUZYB3KKITBHYBLQCLMAUH4KATZOA7MP3EOSCEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.009688859 (18h5m46.968786106s) [noise=true , address=104.36.82.17:4339 , piece_id=EH45THVQ5VWNWT2IZQNFCNHJHVKFVV2LUQLSOU6SY2U3XFTAZ7EA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020012296 (18h5m46.979109545s) [noise=true , address=104.36.82.37:4673 , piece_id=2KFXI33RQ7KENY4SYRW5BXKNBGIZEFGUMCLX3L3EU5V4XILN6CWQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020038982 (18h5m46.979136229s) [seg_pos={0 0}, upload=18850]: Segment reached optimal threshold of 14 pieces. Aug 9 11:31:25.020252359 (18h5m46.979349611s) [noise=true , address=104.36.82.23:4446 , piece_id=2FNVFNB4FWHOXX2M3457YLBUXNNRBA7GO2J4LOZKZ555LF7DTOEQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020399383 (18h5m46.979496618s) [noise=true , address=104.36.82.39:4752 , piece_id=TXMONFM3KQFR54GBH4LWZWSLGTFGGCOQY7MQBODUT2KOVOT6HAOA, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qoB7SaK4p6GwyHAKnheGGuMvQFzA3cerZL5nx7KnpazomyCTT): protocol: expected piece hash; context canceled; EOF Aug 9 11:31:25.020556941 (18h5m46.979654176s) [noise=true , address=104.36.82.16:4314 , piece_id=CWQJ7M2AJ7GUQJVBOUEROTQTSU3M4JMH6DH5UQMYTM6P42HIUNSQ, seg_pos={0 0}, upload=18850]: Done uploading piece. err: <nil> Aug 9 11:31:25.020613885 (18h5m46.97971112s) [noise=true , address=64.49.13.138:4488 , piece_id=SDW64HD54FZQEFPXVXNZIOUUYTNBHC5AFUU5HQRANSHMYSYZT62Q, seg_pos={0 0}, upload=18850]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mQqzHsTJvR5wHpEGXDixk3NcRuBKxY2cL82eJMHidcxFiPCZb): protocol: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49200->64.49.13.138:4488: use of closed network connection Aug 9 11:31:25.020632589 (18h5m46.979729845s) [seg_pos={0 0}, upload=18850]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 9 11:31:25.020719875 (18h5m46.979817126s) [upload=18850]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 11:31:25.020725423 (18h5m46.979822669s) [upload=18850]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:31:25.498139632 (18h5m47.457236878s) [upload=18850]: Done uploading segments... Aug 9 11:31:25.498160816 (18h5m47.457258063s) [upload=18850]: Upload finished. err: <nil> Duration: 2.382855585s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 957.155839ms FAIL (2025-08-09 11:59:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 957.183962ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 938.98425ms FAIL (2025-08-09 11:58:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 0d8b7cbee96cf7bb3bf9e40650415f6b91b1de2ca13ac9f4860dad35e96ca17c != before ea1a1839f95f99bddcad2466b4954b39247917e87ad6c595705ffc85efab6c2e ======LOGS====== Aug 9 11:36:16.648290742 (18h10m38.607387987s) [upload=18930]: Starting upload Aug 9 11:36:16.648344863 (18h10m38.607442108s) [upload=18930]: Done starting upload Aug 9 11:36:16.648512996 (18h10m38.607610242s) [upload=18930]: Uploading segments... Aug 9 11:36:16.648608364 (18h10m38.607705619s) [upload=18930]: Got next segment. Inline: true Aug 9 11:36:16.648617609 (18h10m38.607714858s) [upload=18930]: Waiting for error group managing segments... Aug 9 11:36:16.648676479 (18h10m38.607773728s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 11:36:16.648684263 (18h10m38.607781515s) [upload=18930]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 11:36:16.648688393 (18h10m38.60778564s) [upload=18930]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 11:36:16.805840324 (18h10m38.764937577s) [upload=18930]: Done uploading segments... Aug 9 11:36:16.805881159 (18h10m38.764978413s) [upload=18930]: Upload finished. err: <nil> Duration:* 939.01647ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)