Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 16.812702085s FAIL (2025-07-09 21:08:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 16.812733951s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 22.602044609s FAIL (2025-07-09 21:08:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 22.602106922s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.363082202s FAIL (2025-07-09 21:08:03 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.363119643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.387703477s FAIL (2025-07-09 21:07:40 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.387757701s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 35.032649409s FAIL (2025-07-09 21:07:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 35.03270893s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 34.997925314s FAIL (2025-07-09 21:07:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 34.997949508s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.502136248s FAIL (2025-07-09 21:07:02 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.502168454s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.336504041s FAIL (2025-07-09 21:06:34 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.336531925s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 35.852171239s FAIL (2025-07-09 21:06:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 35.852197026s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 24.952068722s FAIL (2025-07-09 21:06:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 24.952081437s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 47.532538035s FAIL (2025-07-09 21:05:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 47.532563562s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 17.064690182s FAIL (2025-07-09 21:05:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.064717475s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.346389782s FAIL (2025-07-09 21:05:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.346427688s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 28.821344324s FAIL (2025-07-09 21:05:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 28.821373202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 59.834634326s FAIL (2025-07-09 21:05:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 59.834664039s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.352905136s FAIL (2025-07-09 21:04:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.352933941s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 18.48842745s FAIL (2025-07-09 21:04:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.488443205s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 29.690305661s FAIL (2025-07-09 21:04:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 29.690329053s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.430348555s FAIL (2025-07-09 21:04:12 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.430397887s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 34.832749544s FAIL (2025-07-09 21:03:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 34.832778364s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 12.0042022s FAIL (2025-07-09 21:03:10 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 12.004228261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 18.732929123s FAIL (2025-07-09 21:03:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 18.732955861s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 16.636061689s FAIL (2025-07-09 21:02:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.636079457s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 31.178888811s FAIL (2025-07-09 21:02:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 31.178916255s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.318747518s FAIL (2025-07-09 21:02:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.31877808s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 38.939355027s FAIL (2025-07-09 21:02:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 38.939411942s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.311786283s FAIL (2025-07-09 21:01:57 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.311822314s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 42.735482254s FAIL (2025-07-09 21:01:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 42.735498509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 25.574540641s FAIL (2025-07-09 21:01:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.574573496s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.34586885s FAIL (2025-07-09 21:01:23 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.345919228s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 21.524116277s FAIL (2025-07-09 21:01:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 21.524140473s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 25.730433771s FAIL (2025-07-09 21:01:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.730464465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.326854137s FAIL (2025-07-09 21:00:59 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.327037514s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 23.334309806s FAIL (2025-07-09 21:00:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 23.334345963s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.804787169s FAIL (2025-07-09 21:00:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.804833051s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 25.627915747s FAIL (2025-07-09 20:59:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 25.627942748s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.336707686s FAIL (2025-07-09 20:59:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.336743399s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 28.16853439s FAIL (2025-07-09 20:59:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 28.168551394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 27.661779487s FAIL (2025-07-09 20:59:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 27.66180325s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.596136629s FAIL (2025-07-09 20:59:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.596168863s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [X..............................................XX...........] duration: 38.482401914s FAIL (2025-07-09 20:59:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 38.482426327s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.801028118s FAIL (2025-07-09 20:58:57 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 5.801061836s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 30.419709807s FAIL (2025-07-09 20:58:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 30.419808795s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 2.608121673s FAIL (2025-07-09 20:58:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.608153229s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.31380105s FAIL (2025-07-09 20:58:24 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.3138341s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 14.458584774s FAIL (2025-07-09 20:58:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 14.458617712s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.X..............................................XX..........] duration: 39.06902222s FAIL (2025-07-09 20:58:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 39.069045048s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.310760064s FAIL (2025-07-09 20:58:02 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.310831335s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 41.153244039s FAIL (2025-07-09 20:57:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== ul 9 20:53:05.181732432 (395h42m5.065146066s) [upload=316437]: Upload finished. err: <nil> Jul 9 20:53:46.239759806 (395h42m46.123173434s) [upload=316447]: Starting upload Jul 9 20:53:46.239879159 (395h42m46.123292791s) [upload=316447]: Done starting upload Jul 9 20:53:46.240039985 (395h42m46.123453618s) [upload=316447]: Uploading segments... Jul 9 20:53:46.240311285 (395h42m46.123724915s) [upload=316447]: Got next segment. Inline: false Jul 9 20:53:46.240364403 (395h42m46.123778042s) [upload=316447]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:53:46.240374251 (395h42m46.123787889s) [upload=316447]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:53:46.440120932 (395h42m46.323534548s) [seg_pos={0 0}, upload=316447]: Begin upload segment... Jul 9 20:53:46.440179950 (395h42m46.323593568s) [seg_pos={0 0}, upload=316447]: Done begin upload segment. Jul 9 20:53:46.440217375 (395h42m46.323631001s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440650993 (395h42m46.32406462s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.440924286 (395h42m46.324337911s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441186006 (395h42m46.324599632s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441444482 (395h42m46.324858108s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.441690913 (395h42m46.325104542s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442118579 (395h42m46.325532246s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442302181 (395h42m46.32571581s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.442705594 (395h42m46.326119222s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443169151 (395h42m46.326582779s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443516420 (395h42m46.326930048s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.443951987 (395h42m46.327365616s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444246383 (395h42m46.32766001s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444526835 (395h42m46.327940466s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.444714260 (395h42m46.328127876s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445082976 (395h42m46.328496603s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.445455193 (395h42m46.328868819s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:46.446344122 (395h42m46.329757751s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Uploading piece... Jul 9 20:53:47.020800550 (395h42m46.90421418s) [upload=316447]: Next returned nil segment Jul 9 20:53:47.020819787 (395h42m46.904233401s) [upload=316447]: Waiting for error group managing segments... Jul 9 20:53:47.156598268 (395h42m47.04001189s) [noise=true , address=77.90.31.12:4603 , piece_id=RB6XNVBOF373CV2ORP736LESCVTPEJIXEZJQDL3MBIDQFLQ2AXLQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.159389097 (395h42m47.042802716s) [noise=true , address=77.90.31.11:4582 , piece_id=FYEQYVURALHPDG6JUOA2SDXTTVTXPA6CXEJ2X7D7BFS2B7JJKW4Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.160887797 (395h42m47.044301415s) [noise=true , address=77.90.31.18:4824 , piece_id=LLO7JQBAN7BCSNJVXOVSE4YQCBTU74F237MBZRJ6NRBAPAKIBH3Q, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.163302365 (395h42m47.046715982s) [noise=true , address=77.90.31.21:4911 , piece_id=II36UAEZ7F7LTRKKMJPGKEIAR7XLCOQOAGG4CYH3FOZ6VUHVTNVA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.167069778 (395h42m47.050483398s) [noise=true , address=77.90.31.13:4630 , piece_id=L7QDIQGANAS6DNSQCVOAVRI7N74IWUODKS7SF7BLWDEKNAKO6QEA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.168030609 (395h42m47.051444228s) [noise=true , address=77.90.31.15:4702 , piece_id=XSHZPS6CN3AQQZ7GFJVJJM4YFWJODUWEPOT6HKQ3NFSHOBSUPWGA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:47.383117245 (395h42m47.266530862s) [noise=true , address=79.127.220.98:4423 , piece_id=2ZQKHDZEWY5EGUWLRLQBMEUWTWZUO2MK3A5VGUDTTGZDNOBADXPQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:48.832058921 (395h42m48.71547254s) [noise=true , address=79.127.201.215:4348 , piece_id=P643SCY27W5ZOBMVT3LD6LZD5XTP3OW5OYGX5TD32T5JKVXD7HHA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:53:49.721740279 (395h42m49.605153896s) [noise=true , address=45.91.95.2:4002 , piece_id=OLCA5YKYX4AYB3BFE4SY2ZBGQUDVI3WE7R7CDAABMENLFLQL46OQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1ceeydU13HxVgA5trT5MumrQoCYKWURrkoTZLPXBXjGiEC23W9, address:45.91.95.2:4002): protocol: storage node overloaded, request limit: 2000 Jul 9 20:53:58.022618980 (395h42m57.9060326s) [noise=true , address=79.127.219.38:4030 , piece_id=TM344C6YJKYD5JDEHF7CY4HQA4WBEQUH6NSFPRC3PS62EPDRAIJQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:03.273249375 (395h43m3.156663002s) [noise=true , address=104.36.82.18:4387 , piece_id=FHI4YIRE5QX5TGS66YP2HWEVZKYHK4SP6ZBOJLLTP6Z5JLK6SC5A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:09.469438565 (395h43m9.352852186s) [noise=true , address=104.36.82.4:4833 , piece_id=E6NMZGIVIZVQYBABG6IKSZGLJXEYPPGY6OG4U6M56BTAYYXM3CJA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:10.167123637 (395h43m10.050537256s) [noise=true , address=104.36.82.51:4081 , piece_id=YTR54NHEDMDMRCOIR7Z6SLAJNR4LLAT7SD5I6ZJLKC2RTHFLUDYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:38.676415445 (395h43m38.559829065s) [noise=true , address=104.36.82.17:4354 , piece_id=NBPDHYLJJSGMSQQ6X4GWTAB432KK7ZIBG54VCR36IOV53TPPD7YA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: <nil> Jul 9 20:54:45.960480350 (395h43m45.843894004s) [noise=true , address=104.36.82.60:4347 , piece_id=IUFS5CNRLX74PROKKFT3TD7LNLTCAP4SF2FQQU2PWKF7IFKBH47A, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1CbCAPrrYNvhv8129X18ftZmajktekzBFMu5rMLYpHriJbN5PM, address:104.36.82.60:4347): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960513865 (395h43m45.843927481s) [noise=true , address=45.91.95.3:4015 , piece_id=AIKGX3EM5TAUSTSZAOVYYNEJDRKXNMH2GDR55CHTOKQJQK4JYNYA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1Nr35HEapmku71PcyhfgfDzAR9RwCuz9Q5KAEMj6TTCbrVSe7U, address:45.91.95.3:4015): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960590132 (395h43m45.844003749s) [noise=true , address=104.36.82.19:4427 , piece_id=DZBXNLAHIVL7K54ORKHKYSBEMT4YH3D3XRBCMEZUDJRCIFBFDLPA, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:1NdaXHU5WMvCtca7R6XRycKQGBQio4cikujGTu8TwnyPcGf7uG, address:104.36.82.19:4427): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960601510 (395h43m45.844015126s) [noise=true , address=45.91.95.4:4036 , piece_id=6FQHHGIXR3ZWG7AO25QWYQJPB77WIJVPUDCMILTYDBSXZ7KDQMEQ, seg_pos={0 0}, upload=316447]: Done uploading piece. err: ecclient: upload failed (node:12iVWBkBV5xpsCNemxmJasZtAyWa7L2QxMDyvKK85DcHvnsZ4YS, address:45.91.95.4:4036): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 20:54:45.960620902 (395h43m45.844034518s) [seg_pos={0 0}, upload=316447]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 9 20:54:46.067458842 (395h43m45.950872454s) [upload=316447]: Done uploading segments... Jul 9 20:54:46.067486298 (395h43m45.950899914s) [upload=316447]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 41.153281281s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.297234939s FAIL (2025-07-09 20:57:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== g_pos={0 0}, upload=316367]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:47:36.244291365 (395h36m36.127704992s) [upload=316367]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:47:36.244296534 (395h36m36.12771016s) [upload=316367]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:47:36.398064624 (395h36m36.281478263s) [upload=316367]: Done uploading segments... Jul 9 20:47:36.398088829 (395h36m36.281502458s) [upload=316367]: Upload finished. err: <nil> Jul 9 20:48:05.350156635 (395h37m5.233570264s) [upload=316375]: Starting upload Jul 9 20:48:05.350246294 (395h37m5.233659914s) [upload=316375]: Done starting upload Jul 9 20:48:05.350375398 (395h37m5.233789018s) [upload=316375]: Uploading segments... Jul 9 20:48:05.350561373 (395h37m5.233974991s) [upload=316375]: Got next segment. Inline: false Jul 9 20:48:05.350614545 (395h37m5.23402817s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 20:48:05.350620566 (395h37m5.234034192s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 20:48:05.458460235 (395h37m5.341873852s) [seg_pos={0 0}, upload=316375]: Begin upload segment... Jul 9 20:48:05.458527091 (395h37m5.341940719s) [seg_pos={0 0}, upload=316375]: Done begin upload segment. Jul 9 20:48:05.458556195 (395h37m5.34196983s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458562192 (395h37m5.341975828s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.458933587 (395h37m5.342347216s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459058718 (395h37m5.342472347s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459243031 (395h37m5.34265666s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459440806 (395h37m5.342854435s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459561010 (395h37m5.342974638s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.459964235 (395h37m5.343377871s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460006098 (395h37m5.343419737s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460336395 (395h37m5.343750022s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460346182 (395h37m5.343759815s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460394451 (395h37m5.343808083s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460678150 (395h37m5.344091783s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.460949337 (395h37m5.344362969s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461264345 (395h37m5.344677977s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:05.461558061 (395h37m5.34497169s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Uploading piece... Jul 9 20:48:06.075903844 (395h37m5.95931746s) [upload=316375]: Next returned nil segment Jul 9 20:48:06.075920633 (395h37m5.959334263s) [upload=316375]: Waiting for error group managing segments... Jul 9 20:48:06.193841562 (395h37m6.077255175s) [noise=true , address=104.36.82.12:4150 , piece_id=THNEEDG45WKTOE2UUWNUORE47T6BJUMNIX77CLYIDQCWQCUEWG7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.207913388 (395h37m6.091327015s) [noise=true , address=45.91.116.13:4155 , piece_id=MNGVJN2HXXFHVJ3DAG4UKM6ZUW3FYNS5HCWZFE2ITYIR37LKITYQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.216946791 (395h37m6.100360405s) [noise=true , address=45.91.116.14:4190 , piece_id=HHID2EF5TJQOWIPWVAMWYQ3ZJVWV7QDVVA44G6S4IRSUNZRWILMQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.223899481 (395h37m6.107313111s) [noise=true , address=45.91.116.12:4227 , piece_id=O6PSKTAFNSB7EGS4XYN5JX7FGV7RMNOLVEPLUUBO34OT2BNE3MBQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.224185439 (395h37m6.107599059s) [noise=true , address=104.36.82.10:4078 , piece_id=TWYODK6NJ7NAAUUVTJD7GSO2ZW6ALCZNYMLMWZNM3AXZBRWCIJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.228926526 (395h37m6.112340161s) [noise=true , address=77.90.31.17:4772 , piece_id=2AEBDEJDYC2DLSGEMIYDU7IOBWOSFFGNJD5Z2OWAF7QWB6BB2USA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231175580 (395h37m6.114589217s) [noise=true , address=45.91.116.9:4262 , piece_id=IZHLUIKUZTCNIL2K7V5OCSEIS52STDTXLOMWJIFXHJEX6HBYD7CA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.231929100 (395h37m6.115342718s) [noise=true , address=77.90.31.15:4698 , piece_id=7O2XEHXVN4EPW7OK2IX22MQWYFG25645S5NUQN3NBHW5DPMSX2MA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.234668051 (395h37m6.11808168s) [noise=true , address=77.90.31.11:4553 , piece_id=6BYT5IOCYEYGGDVNODBBLQVFKTOFHZSPV6ZXU2EUY25BF2N2WJFA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.235896082 (395h37m6.119309711s) [noise=true , address=77.90.31.13:4627 , piece_id=ZY7KKGZASUM7HWAMBDCBERUYAYPW6DJFA4VNWN4AXUAFE76XW7WA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:06.245500968 (395h37m6.128914582s) [noise=true , address=104.36.82.11:4115 , piece_id=VI4UWB2ROZGL6P5UZ2ND6MSM3PNRW2OJPVJPXJDVIM3LH55B7R7A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:29.284655472 (395h37m29.168069085s) [noise=true , address=104.36.82.6:4874 , piece_id=7W7GZJLTWU37FH6QNDJMICZ2YNHJ7OVJOEIC2CYO7ID74QOF3U6A, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953757464 (395h37m33.837171094s) [noise=true , address=104.36.82.3:4765 , piece_id=PRPV2SBSURY7V7HRW32BEBQBQ44QVXJC77YRI7N5OEZ2S25T4DQA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: <nil> Jul 9 20:48:33.953797165 (395h37m33.837210795s) [seg_pos={0 0}, upload=316375]: Segment reached optimal threshold of 13 pieces. Jul 9 20:48:33.954000365 (395h37m33.837413985s) [noise=true , address=104.36.82.2:4731 , piece_id=VELITJJTCY2QED7XLBQSYW5JY63LU4WXE25YEGCRNT6Y3PFBQGAQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 9 20:48:33.954159166 (395h37m33.837572794s) [noise=true , address=104.36.82.4:4803 , piece_id=HX2V5HWN4O5QZRAQNHYT5UMXQA57KNAC42EXBWN547GEAUW5KLSQ, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7): protocol: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44968->104.36.82.4:4803: use of closed network connection Jul 9 20:48:33.954230334 (395h37m33.837643962s) [noise=true , address=104.36.82.5:4839 , piece_id=3IDN23C3L3AOOM7XZGGKLGPI5PUYCY5VL76JZKKV2FPZRC4VVPJA, seg_pos={0 0}, upload=316375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60818->104.36.82.5:4839: use of closed network connection Jul 9 20:48:33.954257932 (395h37m33.837671545s) [seg_pos={0 0}, upload=316375]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 20:48:33.954398670 (395h37m33.8378123s) [upload=316375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 20:48:33.954403994 (395h37m33.837817606s) [upload=316375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 20:48:34.117929330 (395h37m34.001342957s) [upload=316375]: Done uploading segments... Jul 9 20:48:34.117948670 (395h37m34.001362287s) [upload=316375]: Upload finished. err: <nil> Duration: 2.297262615s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)