Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 226.670056ms FAIL (2025-07-07 16:17:19 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 226.70108ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.474026923s FAIL (2025-07-07 16:17:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.474053879s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.255234557s FAIL (2025-07-07 16:16:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 1.255347142s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 261.53901ms FAIL (2025-07-07 16:16:51 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 261.721821ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.039036464s FAIL (2025-07-07 16:16:35 UTC) orbiter warning: context deadline exceeded Duration: 1.039104695s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.474263036s FAIL (2025-07-07 16:16:12 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.47429509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 250.802095ms FAIL (2025-07-07 16:16:01 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 250.836263ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 882.8028ms FAIL (2025-07-07 16:15:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 882.839809ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 928.580761ms FAIL (2025-07-07 16:15:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 928.609555ms core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.212342705s FAIL (2025-07-07 16:15:26 UTC) orbiter warning: context deadline exceeded Duration: 1.212415591s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.49622146s FAIL (2025-07-07 16:15:12 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.496251522s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 222.315218ms FAIL (2025-07-07 16:15:02 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 222.367383ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.369546358s FAIL (2025-07-07 16:14:31 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.369575323s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.061718662s FAIL (2025-07-07 16:14:31 UTC) orbiter warning: context deadline exceeded Duration: 1.061750114s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.274088516s FAIL (2025-07-07 16:14:16 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.274142112s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 879.661681ms FAIL (2025-07-07 16:14:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 879.688689ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.426348788s FAIL (2025-07-07 16:13:59 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 2.426387077s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 206.992757ms FAIL (2025-07-07 16:13:56 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 207.024713ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.184942427s FAIL (2025-07-07 16:13:52 UTC) orbiter warning: context deadline exceeded Duration: 1.184978307s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 4.104848283s FAIL (2025-07-07 16:13:42 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 4.104881268s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.00072811s FAIL (2025-07-07 16:13:35 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 54m5.522882535s) [upload=274201]: Starting upload Jul 7 16:05:05.639568228 (342h54m5.522981865s) [upload=274201]: Done starting upload Jul 7 16:05:05.639650104 (342h54m5.523063722s) [upload=274201]: Uploading segments... Jul 7 16:05:05.639694258 (342h54m5.523107896s) [upload=274201]: Got next segment. Inline: true Jul 7 16:05:05.639700053 (342h54m5.523113682s) [upload=274201]: Waiting for error group managing segments... Jul 7 16:05:05.639745740 (342h54m5.523159358s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:05.639751967 (342h54m5.523165587s) [upload=274201]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:05.639756742 (342h54m5.523170373s) [upload=274201]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:05.785101960 (342h54m5.668515598s) [upload=274201]: Done uploading segments... Jul 7 16:05:05.785128979 (342h54m5.668542612s) [upload=274201]: Upload finished. err: <nil> Jul 7 16:05:48.475319131 (342h54m48.358732762s) [upload=274214]: Starting upload Jul 7 16:05:48.475398500 (342h54m48.358812128s) [upload=274214]: Done starting upload Jul 7 16:05:48.475490431 (342h54m48.35890405s) [upload=274214]: Uploading segments... Jul 7 16:05:48.475556523 (342h54m48.358970208s) [upload=274214]: Got next segment. Inline: true Jul 7 16:05:48.475562302 (342h54m48.358975923s) [upload=274214]: Waiting for error group managing segments... Jul 7 16:05:48.475623267 (342h54m48.359036882s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:05:48.475628904 (342h54m48.359042522s) [upload=274214]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:05:48.475632467 (342h54m48.359046083s) [upload=274214]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:05:48.630081298 (342h54m48.513494937s) [upload=274214]: Done uploading segments... Jul 7 16:05:48.630109964 (342h54m48.513523594s) [upload=274214]: Upload finished. err: <nil> Jul 7 16:06:53.603873610 (342h55m53.487287231s) [upload=274229]: Starting upload Jul 7 16:06:53.603978758 (342h55m53.487392376s) [upload=274229]: Done starting upload Jul 7 16:06:53.604118721 (342h55m53.487532356s) [upload=274229]: Uploading segments... Jul 7 16:06:53.604216612 (342h55m53.48763025s) [upload=274229]: Got next segment. Inline: true Jul 7 16:06:53.604225427 (342h55m53.48763906s) [upload=274229]: Waiting for error group managing segments... Jul 7 16:06:53.604290720 (342h55m53.487704354s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:06:53.604298201 (342h55m53.487711837s) [upload=274229]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:06:53.604302426 (342h55m53.487716056s) [upload=274229]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:06:53.769943017 (342h55m53.653356649s) [upload=274229]: Done uploading segments... Jul 7 16:06:53.769967128 (342h55m53.653380757s) [upload=274229]: Upload finished. err: <nil> Jul 7 16:07:09.781180971 (342h56m9.664594606s) [upload=274234]: Starting upload Jul 7 16:07:09.781273414 (342h56m9.664687043s) [upload=274234]: Done starting upload Jul 7 16:07:09.781397712 (342h56m9.664811327s) [upload=274234]: Uploading segments... Jul 7 16:07:09.781473011 (342h56m9.664886649s) [upload=274234]: Got next segment. Inline: true Jul 7 16:07:09.781479917 (342h56m9.664893529s) [upload=274234]: Waiting for error group managing segments... Jul 7 16:07:09.781548856 (342h56m9.66496249s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:09.781555876 (342h56m9.664969513s) [upload=274234]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:09.781559345 (342h56m9.664972991s) [upload=274234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:09.929512930 (342h56m9.812926557s) [upload=274234]: Done uploading segments... Jul 7 16:07:09.929537978 (342h56m9.812951604s) [upload=274234]: Upload finished. err: <nil> Jul 7 16:07:58.898435851 (342h56m58.781849466s) [upload=274243]: Starting upload Jul 7 16:07:58.898503422 (342h56m58.781917051s) [upload=274243]: Done starting upload Jul 7 16:07:58.898569319 (342h56m58.781982951s) [upload=274243]: Uploading segments... Jul 7 16:07:58.898619425 (342h56m58.782033064s) [upload=274243]: Got next segment. Inline: true Jul 7 16:07:58.898626236 (342h56m58.782039867s) [upload=274243]: Waiting for error group managing segments... Jul 7 16:07:58.898668444 (342h56m58.782082061s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:07:58.898674730 (342h56m58.782088346s) [upload=274243]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:07:58.898678291 (342h56m58.782091917s) [upload=274243]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:07:59.053430969 (342h56m58.936844605s) [upload=274243]: Done uploading segments... Jul 7 16:07:59.053453807 (342h56m58.936867443s) [upload=274243]: Upload finished. err: <nil> Jul 7 16:08:31.351020607 (342h57m31.234434225s) [upload=274251]: Starting upload Jul 7 16:08:31.351085933 (342h57m31.234499551s) [upload=274251]: Done starting upload Jul 7 16:08:31.351160733 (342h57m31.234574354s) [upload=274251]: Uploading segments... Jul 7 16:08:31.351204405 (342h57m31.23461804s) [upload=274251]: Got next segment. Inline: true Jul 7 16:08:31.351209737 (342h57m31.234623354s) [upload=274251]: Waiting for error group managing segments... Jul 7 16:08:31.351262285 (342h57m31.23467592s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:08:31.351270372 (342h57m31.234684008s) [upload=274251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:08:31.351274789 (342h57m31.234688412s) [upload=274251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:08:31.666975003 (342h57m31.550388636s) [upload=274251]: Done uploading segments... Jul 7 16:08:31.666996840 (342h57m31.550410468s) [upload=274251]: Upload finished. err: <nil> Jul 7 16:09:14.540332978 (342h58m14.423746604s) [upload=274260]: Starting upload Jul 7 16:09:14.540536079 (342h58m14.423949709s) [upload=274260]: Done starting upload Jul 7 16:09:14.540686691 (342h58m14.424100307s) [upload=274260]: Uploading segments... Jul 7 16:09:14.540748119 (342h58m14.424161745s) [upload=274260]: Got next segment. Inline: true Jul 7 16:09:14.540754037 (342h58m14.424167655s) [upload=274260]: Waiting for error group managing segments... Jul 7 16:09:14.540824614 (342h58m14.424238244s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:09:14.540831836 (342h58m14.424245463s) [upload=274260]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:09:14.540835228 (342h58m14.424248841s) [upload=274260]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:09:14.727704710 (342h58m14.611118326s) [upload=274260]: Done uploading segments... Jul 7 16:09:14.727724336 (342h58m14.611137952s) [upload=274260]: Upload finished. err: <nil> Jul 7 16:10:21.056234621 (342h59m20.939648238s) [upload=274275]: Starting upload Jul 7 16:10:21.056351046 (342h59m20.939764673s) [upload=274275]: Done starting upload Jul 7 16:10:21.056515193 (342h59m20.939928824s) [upload=274275]: Uploading segments... Jul 7 16:10:21.056620796 (342h59m20.940034427s) [upload=274275]: Got next segment. Inline: true Jul 7 16:10:21.056626145 (342h59m20.940039757s) [upload=274275]: Waiting for error group managing segments... Jul 7 16:10:21.056667856 (342h59m20.940081469s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:10:21.056672528 (342h59m20.940086139s) [upload=274275]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:10:21.056674839 (342h59m20.940088466s) [upload=274275]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:21.196367062 (342h59m21.079780688s) [upload=274275]: Done uploading segments... Jul 7 16:10:21.196387481 (342h59m21.079801106s) [upload=274275]: Upload finished. err: <nil> Jul 7 16:11:09.492199169 (343h0m9.375612796s) [upload=274286]: Starting upload Jul 7 16:11:09.492276634 (343h0m9.37569026s) [upload=274286]: Done starting upload Jul 7 16:11:09.492436918 (343h0m9.375850548s) [upload=274286]: Uploading segments... Jul 7 16:11:09.492547571 (343h0m9.375961207s) [upload=274286]: Got next segment. Inline: true Jul 7 16:11:09.492555201 (343h0m9.375968838s) [upload=274286]: Waiting for error group managing segments... Jul 7 16:11:09.492621267 (343h0m9.376034912s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:11:09.492629345 (343h0m9.376042984s) [upload=274286]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:11:09.492633208 (343h0m9.376046841s) [upload=274286]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:11:09.641767157 (343h0m9.525180813s) [upload=274286]: Done uploading segments... Jul 7 16:11:09.641793156 (343h0m9.525206794s) [upload=274286]: Upload finished. err: <nil> Jul 7 16:12:07.350661891 (343h1m7.234075507s) [upload=274298]: Starting upload Jul 7 16:12:07.350757521 (343h1m7.234171134s) [upload=274298]: Done starting upload Jul 7 16:12:07.350854330 (343h1m7.234267956s) [upload=274298]: Uploading segments... Jul 7 16:12:07.350974984 (343h1m7.234388663s) [upload=274298]: Got next segment. Inline: true Jul 7 16:12:07.350983684 (343h1m7.234397405s) [upload=274298]: Waiting for error group managing segments... Jul 7 16:12:07.351058128 (343h1m7.234471761s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 7 16:12:07.351066068 (343h1m7.2344797s) [upload=274298]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jul 7 16:12:07.351070783 (343h1m7.234484414s) [upload=274298]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:12:07.582967458 (343h1m7.466381088s) [upload=274298]: Done uploading segments... Jul 7 16:12:07.582987682 (343h1m7.466401298s) [upload=274298]: Upload finished. err: <nil> Duration: 30.00075905s core uplink us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 168.386917ms FAIL (2025-07-07 16:13:13 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 168.424679ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 40.309595086s FAIL (2025-07-07 16:13:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 40.309620148s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.004942581s FAIL (2025-07-07 16:13:04 UTC) orbiter warning: context deadline exceeded Duration: 1.00501832s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 207.029958ms FAIL (2025-07-07 16:12:54 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 207.063167ms core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000571841s FAIL (2025-07-07 16:12:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== ul 7 16:10:15.401196269 (342h59m15.284609901s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401254233 (342h59m15.284667919s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.401746737 (342h59m15.285160367s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.403249311 (342h59m15.286662944s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404006550 (342h59m15.287420178s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.404836812 (342h59m15.288250444s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.405448499 (342h59m15.288862129s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406095403 (342h59m15.289509033s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.406673344 (342h59m15.290086985s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407021959 (342h59m15.290435591s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407349365 (342h59m15.290762998s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407632637 (342h59m15.291046268s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.407986333 (342h59m15.291399965s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408319075 (342h59m15.291732707s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.408674224 (342h59m15.292087852s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409017001 (342h59m15.29243063s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409173254 (342h59m15.292586893s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.409582705 (342h59m15.292996334s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Uploading piece... Jul 7 16:10:15.952377536 (342h59m15.835791154s) [upload=274274]: Next returned nil segment Jul 7 16:10:15.952508668 (342h59m15.835922299s) [upload=274274]: Waiting for error group managing segments... Jul 7 16:10:16.100323243 (342h59m15.983736874s) [noise=true , address=104.36.82.34:4578 , piece_id=RAAVA7T22V5LO4JJTILDX4KU5M7BTUMD7LQ7L3UC64T5SSPSMRQQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.101418310 (342h59m15.984831944s) [noise=true , address=77.90.31.17:4798 , piece_id=EFM6O36ZKSWIG72XLYE5VCTVYXUDLSGLHLUQSK6P4UWXGJWAADMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113573269 (342h59m15.996986889s) [noise=true , address=77.90.31.15:4709 , piece_id=LNZVJYZARI5UPGKBZODU3XADYA7IDITZOM7R6U3QTDB3SOSXLUMA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.113714978 (342h59m15.997128596s) [noise=true , address=77.90.31.21:4900 , piece_id=7T3VJNRVJAJ6JVSMWZSSMJD6YALSKBLYNZOHEOX5A3ERGEBHWNKA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.116007361 (342h59m15.999420981s) [noise=true , address=104.36.82.18:4399 , piece_id=2G5D3HQXKX732YNBKT7EYTV267C7RGVTGAMJP2OVZFCZMEOKMFYA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.119591937 (342h59m16.003005565s) [noise=true , address=77.90.31.10:4548 , piece_id=WKQYY6Y4AZLS44CGN3B756J66MELCC55AJQ6Y5ZLDTRH5XRITTPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.120604019 (342h59m16.004017648s) [noise=true , address=77.90.31.12:4596 , piece_id=RXS27YHXESQTSQI53KTZ3QDNGOQ2NCAWD6ESRHATTDWXFJR6WLGQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.131637022 (342h59m16.01505064s) [noise=true , address=77.90.31.18:4816 , piece_id=TYDKNSSMJ67YYWX2VR4FHHHFGQYODJX6B7FBRSBFNOU5NSE2C2HQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.139805681 (342h59m16.0232193s) [noise=true , address=104.36.82.20:4441 , piece_id=43QM3YT42C27H2IXSF6M52IYJIKMXMBK644XHCD7PDBZH53RZE3A, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.140732732 (342h59m16.024146362s) [noise=true , address=104.36.82.49:4988 , piece_id=VRIJUNNQJCT5RJVKKNFNBBKSQECSNOEXEZIKJ6XTBVUPYYOJTEYQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.143741256 (342h59m16.027154876s) [noise=true , address=104.36.82.39:4736 , piece_id=QOWJJPDR2FRT5DJZJCDM2IPSQIGQTAEHS2QSNI65VAVNNANU2ZGA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.148352306 (342h59m16.031765926s) [noise=true , address=104.36.82.23:4480 , piece_id=Q3NKSFM63CQ2XROU4LHGY6UB4ZAXJWIM6WJ636T6XLBJJJLWMULA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.151040309 (342h59m16.03445394s) [noise=true , address=45.91.116.5:4315 , piece_id=Z2XTS5CWWI5QYG7M4UEIYP7BWVGNBNZSQEHHSKRV76E5WGOAODPA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163822133 (342h59m16.04723575s) [noise=true , address=104.36.82.44:4884 , piece_id=OZWCZN5HXM65RP7O5IMLYL2MU3YUDNSOCEEIILWL7XVNNFUCXCWA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: <nil> Jul 7 16:10:16.163847541 (342h59m16.047261167s) [seg_pos={0 0}, upload=274274]: Segment reached optimal threshold of 14 pieces. Jul 7 16:10:16.164116449 (342h59m16.047530075s) [noise=true , address=64.49.13.154:4929 , piece_id=AL3QF3GNRYE645VJS2TLKND6GEEHYOBLEWTM4ORCF5FW3M4EY33Q, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tY9qdfxejitZBjGRg6r4geETX9wwdUbMCvG2zXQDWnGf9Z5gd): protocol: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:43352->64.49.13.154:4929: use of closed network connection Jul 7 16:10:16.164224185 (342h59m16.047637812s) [noise=true , address=77.90.31.13:4634 , piece_id=Z4ZXPPRRABJKCZGFCK4QJX73FHYTYMIGT5KBPSR27UQ5IPOYWZDA, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Jul 7 16:10:16.165321770 (342h59m16.048735403s) [noise=true , address=193.149.164.143:4693 , piece_id=NKRFOSIKKOH43W3VHTFD3OPACUZEZFQXLPTXBZ5VC7FI6YVWNMMQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LHoCE9CnM32tbMNqAB7HpmArSVLTycv9o6kSSFNUJsoDeERkQ): protocol: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:44756->193.149.164.143:4693: use of closed network connection Jul 7 16:10:16.165483316 (342h59m16.048896949s) [noise=true , address=156.146.43.225:4461 , piece_id=IHWWVXTHYB7LWPN3XJAE4NOERKWXJ75VEMZGU7QZIGLGBNTY3NAQ, seg_pos={0 0}, upload=274274]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1D4k7wnNtivVkHko5CSBuZH6zqMuFyfFfshkZwEPqnmpLads49): protocol: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42254->156.146.43.225:4461: use of closed network connection Jul 7 16:10:16.165501544 (342h59m16.048915175s) [seg_pos={0 0}, upload=274274]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 7 16:10:16.165616538 (342h59m16.049030167s) [upload=274274]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 7 16:10:16.165622460 (342h59m16.049036093s) [upload=274274]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 7 16:10:16.314345828 (342h59m16.197759461s) [upload=274274]: Done uploading segments... Jul 7 16:10:16.314370507 (342h59m16.197784126s) [upload=274274]: Upload finished. err: <nil> Duration: 1m0.000606786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.10801938s FAIL (2025-07-07 16:12:11 UTC) orbiter warning: context deadline exceeded Duration: 1.108054961s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 186.724921ms FAIL (2025-07-07 16:12:05 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 186.79245ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.392378931s FAIL (2025-07-07 16:11:56 UTC) orbiter warning: context deadline exceeded Duration: 1.392431718s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 213.428484ms FAIL (2025-07-07 16:11:40 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 213.45731ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 1.251640217s FAIL (2025-07-07 16:11:01 UTC) orbiter warning: context deadline exceeded Duration: 1.251678644s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 284.857498ms FAIL (2025-07-07 16:10:33 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 284.900533ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 193.174877ms FAIL (2025-07-07 16:10:06 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 193.22674ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.654185836s FAIL (2025-07-07 16:09:48 UTC) orbiter warning: context deadline exceeded Duration: 1.654224144s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 622.556888ms FAIL (2025-07-07 16:09:36 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 622.609603ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 1.555985686s FAIL (2025-07-07 16:08:41 UTC) orbiter warning: context deadline exceeded Duration: 1.556023228s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 638.366055ms FAIL (2025-07-07 16:08:32 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 638.398718ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 1.218363758s FAIL (2025-07-07 16:07:44 UTC) orbiter warning: context deadline exceeded Duration: 1.218399708s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 204.651022ms FAIL (2025-07-07 16:07:40 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 204.681586ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 1.515737478s FAIL (2025-07-07 16:06:52 UTC) orbiter warning: context deadline exceeded Duration: 1.515789249s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 214.28635ms FAIL (2025-07-07 16:06:52 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 214.564435ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.382998754s FAIL (2025-07-07 16:06:30 UTC) orbiter warning: context deadline exceeded Duration: 1.383044283s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 199.723467ms FAIL (2025-07-07 16:06:20 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 199.828191ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 229.717703ms FAIL (2025-07-07 16:05:59 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 229.785029ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 195.632104ms FAIL (2025-07-07 16:05:23 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 195.699038ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 1.395401809s FAIL (2025-07-07 16:05:20 UTC) orbiter warning: context deadline exceeded Duration: 1.395443598s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.249225707s FAIL (2025-07-07 16:05:01 UTC) orbiter warning: context deadline exceeded Duration: 1.249269409s core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 193.907045ms FAIL (2025-07-07 16:04:58 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 193.954366ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 237.147436ms FAIL (2025-07-07 16:03:57 UTC) orbiter warning: uplink: bucket: metaclient: context deadline exceeded Duration: 237.17981ms core uplink eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.373269725s FAIL (2025-07-07 16:03:50 UTC) orbiter warning: context deadline exceeded Duration: 1.373369274s core uplink eu1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)