Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.331266391s FAIL (2025-07-07 16:28:32 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.331294978s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 860.345414ms FAIL (2025-07-07 16:28:10 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 860.381762ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.04721798s FAIL (2025-07-07 16:28:07 UTC) orbiter warning: context deadline exceeded Duration: 1.047253464s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 191.352724ms FAIL (2025-07-07 16:28:00 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 191.384977ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 933.081332ms FAIL (2025-07-07 16:27:49 UTC) orbiter warning: context deadline exceeded Duration: 933.123006ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.314768073s FAIL (2025-07-07 16:27:35 UTC) orbiter warning: context deadline exceeded Duration: 1.314804275s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.444801353s FAIL (2025-07-07 16:27:34 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.444832771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 883.907053ms FAIL (2025-07-07 16:27:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 883.933275ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 197.7569ms FAIL (2025-07-07 16:27:13 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 197.797692ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.235500423s FAIL (2025-07-07 16:27:08 UTC) orbiter warning: context deadline exceeded Duration: 1.235593826s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.407038814s FAIL (2025-07-07 16:26:48 UTC) orbiter warning: context deadline exceeded Duration: 1.407074359s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.126816552s FAIL (2025-07-07 16:26:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 1.126852238s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.312887211s FAIL (2025-07-07 16:26:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.31291952s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 260.7456ms FAIL (2025-07-07 16:26:09 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 260.789182ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 891.471612ms FAIL (2025-07-07 16:25:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 891.503362ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.12822541s FAIL (2025-07-07 16:25:50 UTC) orbiter warning: context deadline exceeded Duration: 1.128258816s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.302405957s FAIL (2025-07-07 16:25:15 UTC) orbiter warning: context deadline exceeded Duration: 1.302435032s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.596231125s FAIL (2025-07-07 16:25:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.596271726s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 941.636594ms FAIL (2025-07-07 16:25:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 941.669323ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 222.05286ms FAIL (2025-07-07 16:25:04 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 222.090478ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.662086101s FAIL (2025-07-07 16:24:22 UTC) orbiter warning: context deadline exceeded Duration: 1.662146341s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 665.1634ms FAIL (2025-07-07 16:24:22 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 665.192306ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 3.021145323s FAIL (2025-07-07 16:24:17 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 3.021245422s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 904.004252ms FAIL (2025-07-07 16:24:03 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 904.041498ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 750.050593ms FAIL (2025-07-07 16:23:53 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 750.0843ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 891.726129ms FAIL (2025-07-07 16:23:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 891.764657ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.521249118s FAIL (2025-07-07 16:23:42 UTC) orbiter warning: context deadline exceeded Duration: 1.52128901s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 3.303604657s FAIL (2025-07-07 16:23:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 3.303640343s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.685658591s FAIL (2025-07-07 16:23:14 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.685707321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 897.053157ms FAIL (2025-07-07 16:23:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 897.100399ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 348.939835ms FAIL (2025-07-07 16:22:50 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 348.975323ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.446024342s FAIL (2025-07-07 16:22:38 UTC) orbiter warning: context deadline exceeded Duration: 1.446062423s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 906.077657ms FAIL (2025-07-07 16:22:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 906.117932ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.41498774s FAIL (2025-07-07 16:22:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.415017256s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 229.459161ms FAIL (2025-07-07 16:21:58 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 229.49922ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.372349194s FAIL (2025-07-07 16:21:50 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.372474569s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.185722967s FAIL (2025-07-07 16:21:36 UTC) orbiter warning: context deadline exceeded Duration: 1.185757006s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 214.955399ms FAIL (2025-07-07 16:21:27 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 214.981663ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 925.539114ms FAIL (2025-07-07 16:21:23 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 925.568567ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.651015105s FAIL (2025-07-07 16:20:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.651048798s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.009677092s FAIL (2025-07-07 16:20:37 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 1.009762936s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 263.852963ms FAIL (2025-07-07 16:20:24 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 263.88619ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.214591879s FAIL (2025-07-07 16:20:22 UTC) orbiter warning: context deadline exceeded Duration: 1.214627453s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.40513744s FAIL (2025-07-07 16:20:13 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.405166074s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 910.529124ms FAIL (2025-07-07 16:19:33 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 910.57241ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.566905794s FAIL (2025-07-07 16:19:31 UTC) orbiter warning: context deadline exceeded Duration: 1.566945606s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 210.897922ms FAIL (2025-07-07 16:19:31 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 210.923389ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.783107423s FAIL (2025-07-07 16:19:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.783139242s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.770570467s FAIL (2025-07-07 16:18:51 UTC) orbiter warning: context deadline exceeded Duration: 1.770603614s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 211.440489ms FAIL (2025-07-07 16:18:46 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 211.494178ms core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)