Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.689704995s FAIL (2025-10-06 16:29:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.68973381s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.02352849s FAIL (2025-10-06 16:29:06 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.023558277s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.526579504s FAIL (2025-10-06 16:28:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.526607968s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.023752474s FAIL (2025-10-06 16:28:25 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.023794375s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.55287754s FAIL (2025-10-06 16:28:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.552919949s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.161743885s FAIL (2025-10-06 16:27:51 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.161774028s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.621764304s FAIL (2025-10-06 16:27:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.621795563s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.279756491s FAIL (2025-10-06 16:26:48 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.279789894s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.563184404s FAIL (2025-10-06 16:26:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.563228398s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.077066296s FAIL (2025-10-06 16:26:18 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.077094664s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.540523584s FAIL (2025-10-06 16:25:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.540553823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.631212161s FAIL (2025-10-06 16:25:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.631249004s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.675405624s FAIL (2025-10-06 16:25:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.675436178s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.248003671s FAIL (2025-10-06 16:25:14 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.248033634s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.120520293s FAIL (2025-10-06 16:24:57 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.120551938s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.1414358s FAIL (2025-10-06 16:24:13 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.141467325s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.513659553s FAIL (2025-10-06 16:24:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.513693422s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.292605564s FAIL (2025-10-06 16:23:23 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.292638625s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.487445641s FAIL (2025-10-06 16:23:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.487476408s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.586036599s FAIL (2025-10-06 16:22:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.586064827s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.55875466s FAIL (2025-10-06 16:22:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.558789599s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.121691216s FAIL (2025-10-06 16:21:56 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.121721705s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.322182941s FAIL (2025-10-06 16:21:14 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.322400828s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.656789484s FAIL (2025-10-06 16:21:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.656889782s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.148321408s FAIL (2025-10-06 16:20:56 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.14836372s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.728885625s FAIL (2025-10-06 16:20:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.728915227s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.05346631s FAIL (2025-10-06 16:20:29 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.053489013s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.479629135s FAIL (2025-10-06 16:19:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.479665902s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.062893786s FAIL (2025-10-06 16:19:25 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.062925763s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.32578189s FAIL (2025-10-06 16:19:04 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.325819714s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.51333201s FAIL (2025-10-06 16:18:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.51336911s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.302374399s FAIL (2025-10-06 16:18:01 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.302406463s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.535436528s FAIL (2025-10-06 16:17:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.535463687s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.964054423s FAIL (2025-10-06 16:17:12 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.964085733s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.524724508s FAIL (2025-10-06 16:17:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.524756654s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.744300403s FAIL (2025-10-06 16:16:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.744413356s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.608574771s FAIL (2025-10-06 16:15:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.60861968s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.206391884s FAIL (2025-10-06 16:15:24 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.206427056s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.15453235s FAIL (2025-10-06 16:15:01 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.154570275s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.579526505s FAIL (2025-10-06 16:14:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.579568028s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.078355431s FAIL (2025-10-06 16:14:05 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.078388431s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.547876276s FAIL (2025-10-06 16:13:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.547907764s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.18187466s FAIL (2025-10-06 16:13:18 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.18191433s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.526540821s FAIL (2025-10-06 16:12:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.526577715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.671895581s FAIL (2025-10-06 16:12:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.671936795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.393046922s FAIL (2025-10-06 16:12:31 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.393100865s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.071841383s FAIL (2025-10-06 16:11:38 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.071893655s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.761983547s FAIL (2025-10-06 16:11:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.762017223s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.691564994s FAIL (2025-10-06 16:11:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.691597814s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.214947086s FAIL (2025-10-06 16:11:05 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.215049175s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)