Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.403938321s FAIL (2025-10-06 16:04:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.403960397s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.544386154s FAIL (2025-10-06 16:04:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.544416044s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.108651207s FAIL (2025-10-06 16:03:30 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.108680343s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.705407997s FAIL (2025-10-06 16:03:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.705443413s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.312897926s FAIL (2025-10-06 16:02:52 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.31292509s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.574498269s FAIL (2025-10-06 16:02:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.574538429s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.310656841s FAIL (2025-10-06 16:02:08 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.310707616s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.59375534s FAIL (2025-10-06 16:01:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.593799914s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.807591756s FAIL (2025-10-06 16:01:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.807622643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.047587138s FAIL (2025-10-06 16:01:15 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.047624689s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.818710613s FAIL (2025-10-06 16:00:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.818817919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.048450728s FAIL (2025-10-06 16:00:14 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.048486478s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.589040713s FAIL (2025-10-06 15:59:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.589072205s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.991917075s FAIL (2025-10-06 15:59:11 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.99197266s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.082581586s FAIL (2025-10-06 15:58:55 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.082616937s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.553444681s FAIL (2025-10-06 15:58:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.553474147s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.498256714s FAIL (2025-10-06 15:58:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.498301982s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.107135274s FAIL (2025-10-06 15:58:00 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.10722981s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.471997984s FAIL (2025-10-06 15:57:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.472028263s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.460592046s FAIL (2025-10-06 15:57:17 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.460626831s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.620931979s FAIL (2025-10-06 15:56:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.62097076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.94316696s FAIL (2025-10-06 15:56:35 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.94319452s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.658537598s FAIL (2025-10-06 15:55:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.65862499s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.201774082s FAIL (2025-10-06 15:55:28 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.201816616s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.596386484s FAIL (2025-10-06 15:54:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.596420485s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.60864702s FAIL (2025-10-06 15:54:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.608727454s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.224371802s FAIL (2025-10-06 15:54:43 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.224471667s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.030445379s FAIL (2025-10-06 15:54:26 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.030474082s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.77371171s FAIL (2025-10-06 15:54:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.773762371s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.627481284s FAIL (2025-10-06 15:53:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.627511312s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.182432783s FAIL (2025-10-06 15:53:22 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.182468586s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.65115179s FAIL (2025-10-06 15:52:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.651192868s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.982890246s FAIL (2025-10-06 15:52:14 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.982921168s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.595694445s FAIL (2025-10-06 15:51:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.595734527s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.545047289s FAIL (2025-10-06 15:51:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.545075043s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 1.987032654s FAIL (2025-10-06 15:51:04 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.987072122s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.174617754s FAIL (2025-10-06 15:50:47 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.174645063s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.545604347s FAIL (2025-10-06 15:50:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.545719576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.175331428s FAIL (2025-10-06 15:50:19 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.175374293s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.028995227s FAIL (2025-10-06 15:49:59 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.029022947s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.530572934s FAIL (2025-10-06 15:49:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.530599438s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 18.608781308s FAIL (2025-10-06 15:49:19 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 18.608814221s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 46.615752977s FAIL (2025-10-06 15:49:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 6 15:48:28.323181802 (314h18m16.126646393s) [upload=276569]: Starting upload Oct 6 15:48:28.323273509 (314h18m16.126738087s) [upload=276569]: Done starting upload Oct 6 15:48:28.323405962 (314h18m16.12687054s) [upload=276569]: Uploading segments... Oct 6 15:48:28.324175266 (314h18m16.127639865s) [upload=276569]: Got next segment. Inline: false Oct 6 15:48:28.324234736 (314h18m16.127699327s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 6 15:48:28.324241207 (314h18m16.127705799s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 6 15:48:28.448109636 (314h18m16.251574228s) [seg_pos={0 0}, upload=276569]: Begin upload segment... Oct 6 15:48:28.448178380 (314h18m16.25164297s) [seg_pos={0 0}, upload=276569]: Done begin upload segment. Oct 6 15:48:28.448207497 (314h18m16.251672094s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.449447618 (314h18m16.252912218s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450272425 (314h18m16.253737017s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.450628085 (314h18m16.254092685s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451240193 (314h18m16.254704783s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.451713584 (314h18m16.255178192s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452167920 (314h18m16.255632514s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452525996 (314h18m16.255990589s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.452921615 (314h18m16.256386209s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453338920 (314h18m16.256803514s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.453701634 (314h18m16.257166228s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454306789 (314h18m16.257771383s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454638758 (314h18m16.25810335s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.454930199 (314h18m16.258394793s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455218559 (314h18m16.258683153s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:28.455494726 (314h18m16.258959319s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Uploading piece... Oct 6 15:48:29.063359790 (314h18m16.866824384s) [upload=276569]: Next returned nil segment Oct 6 15:48:29.063385773 (314h18m16.866850366s) [upload=276569]: Waiting for error group managing segments... Oct 6 15:48:29.207573603 (314h18m17.011038185s) [noise=true , address=45.91.116.4:4938 , piece_id=FIBMEUI7YFRHS64EKIFDJV3UWOZXOXHSU73R3DPFXN4NMHNPGLHA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.210676593 (314h18m17.01414117s) [noise=true , address=45.91.116.13:4153 , piece_id=6JH44B3EJNZM2CXWAFUTURG5CEO2OY2JJF2R7DRVUOUPB42PCCJA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.251463356 (314h18m17.054927939s) [noise=true , address=145.79.227.13:4804 , piece_id=675BJQLFIPFRRMXBCXUWHMBXATHGCJA5J4DSIDPZDVCPO2ZXJJFQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.257610190 (314h18m17.061074773s) [noise=true , address=145.79.227.19:4079 , piece_id=AZYCXUE5VDSQ7I2GMXAZMNDC5AQKPZSQWL6A6HXR5ASX3P3DKTGQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.258398549 (314h18m17.061863126s) [noise=true , address=145.79.227.16:4910 , piece_id=O7N234LDGO4MSYC4RAC3F4WLKZBUBUKMZ2OLCBQU3F2WONHA5COA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.259387548 (314h18m17.062852127s) [noise=true , address=145.79.227.17:4972 , piece_id=4PHRQYM6AWFRTGPT5CTTAACFDIXTPZGMNXOQFKSSKTGVBXPDDB7A, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.262359616 (314h18m17.065824194s) [noise=true , address=145.79.227.7:4294 , piece_id=D334YG6DNFM47LGYUEMT3SU3LF5RCCOFG3NEA5LYAZJQRZSW72EQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.264785106 (314h18m17.068249686s) [noise=true , address=145.79.227.7:4295 , piece_id=ISTGXKKTT6DIP3THAGG26O22MWDJY3HFTK5M3B3UTCS3SFND3QEA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280441321 (314h18m17.083905901s) [noise=true , address=145.79.227.18:4005 , piece_id=57ILRUIJN7GJXVOHFX4UZYWQXLSRSUJUU4OMR4SYAWCL4657MXBQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.280499248 (314h18m17.083963828s) [noise=true , address=145.79.227.3:4151 , piece_id=MBFF7B5GRVYQWKI5H3R3NAZQTSEN3T5BYDKUQGF7PCJZW5HD5PHQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.283118885 (314h18m17.08658347s) [noise=true , address=45.91.116.14:4191 , piece_id=YGUDAIHMX4CVJ3ICZI5VCLAHQWEJHKHYLGZ2APHV3DQ7O4NDVRUA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.284525898 (314h18m17.087990475s) [noise=true , address=77.90.31.15:4699 , piece_id=RRMAEKMQC76UDN57IJPWDJ3EEPGAI3ZV5J5CMEP7ZJVXDTUW6L4Q, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287866544 (314h18m17.091331136s) [noise=true , address=77.90.31.10:4517 , piece_id=Y56CSTTJSWPMQD5OLVIPPW7FH64RAXX2BK3VLHMEMCLLWPGZ6DZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: <nil> Oct 6 15:48:29.287890841 (314h18m17.091355441s) [seg_pos={0 0}, upload=276569]: Segment reached optimal threshold of 13 pieces. Oct 6 15:48:29.288021612 (314h18m17.091486198s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288034623 (314h18m17.091499215s) [noise=true , address=45.91.116.6:4975 , piece_id=6ZCJH35VKWJGGHD7TGKH6BVNALDXYK373KFPXS7PVNCROT6VJTSA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288147194 (314h18m17.091611786s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288156173 (314h18m17.091620769s) [noise=true , address=45.91.116.5:4298 , piece_id=B5GWVEHSNBZ3XVZLJAXTLZQCWESLWT6CLG3N7LYNK42FN2DJ6LZA, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288225500 (314h18m17.09169009s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Oct 6 15:48:29.288234451 (314h18m17.091699042s) [noise=true , address=45.91.116.10:4082 , piece_id=5JKIFAIDG5ZRICLZSJZSBHYTQVS5EI4MBPTLCE5U6DV3QU56OMRQ, seg_pos={0 0}, upload=276569]: Long tail cancellation due to optimal threshold Oct 6 15:48:29.288261295 (314h18m17.091725984s) [seg_pos={0 0}, upload=276569]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 6 15:48:29.288342320 (314h18m17.091806917s) [upload=276569]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 6 15:48:29.288349544 (314h18m17.091814144s) [upload=276569]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 6 15:49:14.636526649 (314h19m2.439991241s) [upload=276569]: Done uploading segments... Oct 6 15:49:14.636563271 (314h19m2.440027867s) [upload=276569]: Upload finished. err: metaclient: context deadline exceeded Duration: 46.615784108s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001087422s FAIL (2025-10-06 15:49:00 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.001145683s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.133636444s FAIL (2025-10-06 14:00:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.133665474s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.183850565s FAIL (2025-10-06 14:00:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.183879278s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.086369341s FAIL (2025-10-06 13:59:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.086406878s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.304273329s FAIL (2025-10-06 13:58:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.304304832s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.144884919s FAIL (2025-10-06 13:57:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.144911617s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.11978848s FAIL (2025-10-06 13:56:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.119828928s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)