Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 885.511546ms FAIL (2025-07-07 16:54:59 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 885.535711ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.371775087s FAIL (2025-07-07 16:54:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.371810715s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.348972398s FAIL (2025-07-07 16:53:58 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.349017773s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 982.977702ms FAIL (2025-07-07 16:53:51 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 983.121339ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 894.331095ms FAIL (2025-07-07 16:53:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 894.38861ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.446304934s FAIL (2025-07-07 16:52:45 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.44633633s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 984.425672ms FAIL (2025-07-07 16:52:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 984.471558ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.517463202s FAIL (2025-07-07 16:51:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.517494568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 868.758288ms FAIL (2025-07-07 16:51:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 868.78909ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 861.897279ms FAIL (2025-07-07 16:50:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 861.93245ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.346941232s FAIL (2025-07-07 16:50:55 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.346969179s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 940.500673ms FAIL (2025-07-07 16:50:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 940.554499ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.608603058s FAIL (2025-07-07 16:49:41 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.608636926s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 870.113769ms FAIL (2025-07-07 16:49:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 870.145079ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.614050603s FAIL (2025-07-07 16:48:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.614083976s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.072836963s FAIL (2025-07-07 16:47:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 1.072865912s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 904.813802ms FAIL (2025-07-07 16:47:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 904.841956ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.547918168s FAIL (2025-07-07 16:47:30 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.547952635s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 969.920028ms FAIL (2025-07-07 16:47:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 969.954848ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.350077139s FAIL (2025-07-07 16:46:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.350105425s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 884.37548ms FAIL (2025-07-07 16:46:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 884.407431ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.411763345s FAIL (2025-07-07 16:46:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.411847765s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.467068171s FAIL (2025-07-07 16:46:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.46710336s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.226183356s FAIL (2025-07-07 16:45:51 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 1.226213394s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.350428402s FAIL (2025-07-07 16:45:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.350459674s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 865.839217ms FAIL (2025-07-07 16:45:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 865.870301ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 893.008208ms FAIL (2025-07-07 16:44:22 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 893.044317ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.968685474s FAIL (2025-07-07 16:44:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.968723241s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 887.984868ms FAIL (2025-07-07 16:43:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 888.016949ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.401045426s FAIL (2025-07-07 16:43:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.401076159s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 987.823743ms FAIL (2025-07-07 16:42:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 987.881171ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.074916692s FAIL (2025-07-07 16:42:24 UTC) orbiter warning: context deadline exceeded Duration: 1.07501696s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.553040385s FAIL (2025-07-07 16:42:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.553071577s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 449.742993ms FAIL (2025-07-07 16:42:04 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 449.776105ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.114967893s FAIL (2025-07-07 16:41:54 UTC) orbiter warning: context deadline exceeded Duration: 1.115016536s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 864.106229ms FAIL (2025-07-07 16:41:36 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 864.132746ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.380761974s FAIL (2025-07-07 16:41:27 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.380858729s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 980.841438ms FAIL (2025-07-07 16:41:15 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 980.871697ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.513080831s FAIL (2025-07-07 16:41:02 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.513145584s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 998.098791ms FAIL (2025-07-07 16:40:57 UTC) orbiter warning: context deadline exceeded Duration: 998.141348ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 182.869584ms FAIL (2025-07-07 16:40:54 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 182.908707ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.190879684s FAIL (2025-07-07 16:40:31 UTC) orbiter warning: context deadline exceeded Duration: 1.190910296s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 857.435553ms FAIL (2025-07-07 16:40:09 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 857.477619ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.502888415s FAIL (2025-07-07 16:40:08 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.502928883s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 179.15554ms FAIL (2025-07-07 16:39:48 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 179.25748ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.111371205s FAIL (2025-07-07 16:39:36 UTC) orbiter warning: context deadline exceeded Duration: 1.111404488s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.805131336s FAIL (2025-07-07 16:39:29 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 1.805160576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 930.465764ms FAIL (2025-07-07 16:39:04 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 930.495902ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 991.691088ms FAIL (2025-07-07 16:38:58 UTC) orbiter warning: context deadline exceeded Duration: 991.724635ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.387707745s FAIL (2025-07-07 16:38:53 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.387743195s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)