Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.791675328s FAIL (2025-10-07 08:31:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.791727872s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.370715526s FAIL (2025-10-07 08:30:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.370755733s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.671669947s FAIL (2025-10-07 08:30:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.671710419s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.779319917s FAIL (2025-10-07 08:29:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.779351994s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.429034661s FAIL (2025-10-07 08:28:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.429067328s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.490856454s FAIL (2025-10-07 08:27:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.490886089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.390377387s FAIL (2025-10-07 08:26:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.390406503s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 10.633498357s FAIL (2025-10-07 08:25:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 10.633535123s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 18.751730465s FAIL (2025-10-07 08:25:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 18.751763871s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.615143327s FAIL (2025-10-07 08:24:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.615174167s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.487128998s FAIL (2025-10-07 08:23:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.487161678s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.297095887s FAIL (2025-10-07 08:23:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.297249824s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.360317536s FAIL (2025-10-07 08:22:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.364765152s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.601561607s FAIL (2025-10-07 08:21:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 2.60159537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.103763957s FAIL (2025-10-07 08:21:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 5s) [upload=291132]: Uploading segments... Oct 7 08:20:45.408939410 (330h50m33.212404008s) [upload=291132]: Got next segment. Inline: false Oct 7 08:20:45.408993506 (330h50m33.212458104s) [upload=291132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:20:45.409000346 (330h50m33.212464942s) [upload=291132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:20:45.519619793 (330h50m33.323084391s) [seg_pos={0 0}, upload=291132]: Begin upload segment... Oct 7 08:20:45.519708774 (330h50m33.323173365s) [seg_pos={0 0}, upload=291132]: Done begin upload segment. Oct 7 08:20:45.519760868 (330h50m33.323225459s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520346701 (330h50m33.323811292s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.520944808 (330h50m33.324409401s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521117895 (330h50m33.324582486s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521407350 (330h50m33.324871951s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521522186 (330h50m33.324986778s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.521962652 (330h50m33.325427267s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522055545 (330h50m33.325520137s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522513190 (330h50m33.325977793s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.522521037 (330h50m33.325985622s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523075320 (330h50m33.326539922s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523103510 (330h50m33.326568105s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523596337 (330h50m33.327060939s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.523621875 (330h50m33.327086466s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.524158097 (330h50m33.327622698s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:45.526079093 (330h50m33.329543684s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Uploading piece... Oct 7 08:20:46.140916541 (330h50m33.944381124s) [upload=291132]: Next returned nil segment Oct 7 08:20:46.140946778 (330h50m33.944412002s) [upload=291132]: Waiting for error group managing segments... Oct 7 08:20:46.271123745 (330h50m34.074588346s) [noise=true , address=45.91.116.5:4298 , piece_id=S3MRJB7XCVO2YEOPZKQOUTCJTBZM4DOQ4U5VC7UPVEVAZIXNHV3Q, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.271414274 (330h50m34.074878859s) [noise=true , address=45.91.116.11:4119 , piece_id=II4L4V7BDAUJQGMMV5XQ5KEJFNXGAPHLO3QXZ32P7GPN7IB6SEWA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.275456129 (330h50m34.078920722s) [noise=true , address=45.91.116.6:4974 , piece_id=QCMYCPCUDCXWBPRJTEUQSAYWTJK6X3BBSZ27QNWBFJWRQFCFVEXQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.284661716 (330h50m34.088126308s) [noise=true , address=77.90.31.14:4661 , piece_id=ULRU2KO74WZP7JIENBOXQRVQBXPVY2DWFAV4NBBTDF74WVRMMACA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.285613800 (330h50m34.089078386s) [noise=true , address=45.91.116.4:4940 , piece_id=RIR7OXRAXOIS2K44OTQUOVBA6EAWDAZL7JEWXIUN3XZ2JYXGFCOQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.287756095 (330h50m34.09122068s) [noise=true , address=77.90.31.18:4807 , piece_id=YHX7XDOCZNKF7GDPDSXG32USWXFOKOYOYYVPVLBTZDCKTFCLSNQQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.296824893 (330h50m34.100289487s) [noise=true , address=77.90.31.15:4699 , piece_id=IZLSA75T5QD67DTAZIC7XHQTXWQFBHO5AHW7SYRDZWIXSIITTM5A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.326180946 (330h50m34.12964553s) [noise=true , address=145.79.227.16:4910 , piece_id=VMOLODUZF2FEJNZ5QDS2QLBBD3FZIS7VIRZTBUW4BLCQ6TS5BHZQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.335625475 (330h50m34.139090079s) [noise=true , address=145.79.227.13:4801 , piece_id=2ZQOQ267CC5EYXLH43ZPGQWOYX4732QSYO6EA47HAVYVEXE2AD7A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.337689760 (330h50m34.14115435s) [noise=true , address=145.79.227.13:4803 , piece_id=U54JBHMWUS7LTSNH3XUDACSQ4C2YQ6TFKD3PASB6NJ7QZRHVRPZA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:20:46.343637412 (330h50m34.147101997s) [noise=true , address=145.79.227.2:4113 , piece_id=HCNPQ4IAAFXVNMLHHAEBIK5IRPIR265OITK3T6MWXDTSJZ5LSU3A, seg_pos={0 0}, upload=291132]: Done uploading piece. err: <nil> Oct 7 08:21:15.127875357 (330h51m2.931339952s) [noise=true , address=145.79.227.7:4293 , piece_id=DR2HQDSCXHZGEDQVL7JCSWUWC5GVKEWAMU3GJB5OARJ6TT6CSQPA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38910->145.79.227.7:4293: use of closed network connection Oct 7 08:21:15.127998971 (330h51m2.931463614s) [noise=true , address=145.79.227.5:4221 , piece_id=YG2FR6Y35M55ETMS2A2PAZEUIIAFZ4JPEBGVZERXLNXU3ZOHX5EA, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34486->145.79.227.5:4221: use of closed network connection Oct 7 08:21:15.128085782 (330h51m2.931550366s) [noise=true , address=145.79.227.3:4150 , piece_id=2PKOHNSYOIYGDIO2CDAOS2X2GZYFMOATN6WPDKWYQMYVKVWWYIBQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40810->145.79.227.3:4150: use of closed network connection Oct 7 08:21:15.128102416 (330h51m2.931567s) [noise=true , address=145.79.227.15:4873 , piece_id=GXRL2RHY3NB6SVPWJ25PSNRPATL33NZSZT6WLHCCQFHFADKSM4TQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46120->145.79.227.15:4873: use of closed network connection Oct 7 08:21:15.128338776 (330h51m2.931803359s) [noise=true , address=145.79.227.6:4257 , piece_id=53MGRPAE2BVNADTXPYBTOPGPTZEH7FYSTNFZQGXJNK6VIBQQYEKQ, seg_pos={0 0}, upload=291132]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48268->145.79.227.6:4257: use of closed network connection Oct 7 08:21:15.128378583 (330h51m2.931843169s) [seg_pos={0 0}, upload=291132]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 08:21:15.229490268 (330h51m3.032954848s) [upload=291132]: Done uploading segments... Oct 7 08:21:15.229520049 (330h51m3.032984641s) [upload=291132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.103817195s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.857866597s FAIL (2025-10-07 07:30:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.857900716s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.639445668s FAIL (2025-10-07 07:29:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.639476432s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.471157496s FAIL (2025-10-07 07:29:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.471187639s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.498642548s FAIL (2025-10-07 07:28:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.498684747s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.251377542s FAIL (2025-10-07 07:26:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.251412045s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.587535715s FAIL (2025-10-07 07:26:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.587614568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.518501583s FAIL (2025-10-07 07:25:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.51859365s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.324919078s FAIL (2025-10-07 07:24:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.32495072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.313686382s FAIL (2025-10-07 07:24:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.31371534s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.869375723s FAIL (2025-10-07 07:23:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.869409489s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.551012019s FAIL (2025-10-07 07:22:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.551039722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.458772724s FAIL (2025-10-07 07:21:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.458815912s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.528624308s FAIL (2025-10-07 07:21:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.528656767s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.573577596s FAIL (2025-10-07 07:20:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.573621514s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.465584093s FAIL (2025-10-07 07:20:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.465618573s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.445125644s FAIL (2025-10-07 07:20:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.44522224s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.346870039s FAIL (2025-10-07 07:19:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.346901239s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.371171582s FAIL (2025-10-07 07:18:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.371206457s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.317983508s FAIL (2025-10-07 07:18:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.318011548s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.434551022s FAIL (2025-10-07 07:17:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.43458317s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.468311827s FAIL (2025-10-07 07:17:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.468342785s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.595674008s FAIL (2025-10-07 07:17:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.595721722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.473679706s FAIL (2025-10-07 07:16:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.473707502s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.558411196s FAIL (2025-10-07 07:15:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.558446304s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.410131162s FAIL (2025-10-07 07:14:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.41017483s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.477590829s FAIL (2025-10-07 07:13:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.477622566s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.435754906s FAIL (2025-10-07 07:13:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.435812397s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.342655659s FAIL (2025-10-07 07:13:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.34268405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.464190228s FAIL (2025-10-07 07:12:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.464245618s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.656737839s FAIL (2025-10-07 07:11:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.656761509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.388938524s FAIL (2025-10-07 07:10:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.388970734s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.33816835s FAIL (2025-10-07 07:09:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.338197735s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.36076554s FAIL (2025-10-07 07:08:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.360796006s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.526402256s FAIL (2025-10-07 07:08:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.526435883s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.281421382s FAIL (2025-10-07 07:08:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 46 (329h16m46.051786948s) [upload=289757]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:46:58.248331649 (329h16m46.051796253s) [upload=289757]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:46:58.429753869 (329h16m46.233218465s) [seg_pos={0 0}, upload=289757]: Begin upload segment... Oct 7 06:46:58.429829794 (329h16m46.233294387s) [seg_pos={0 0}, upload=289757]: Done begin upload segment. Oct 7 06:46:58.430224817 (329h16m46.233689418s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.430360623 (329h16m46.233825224s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431087046 (329h16m46.234551641s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.431878474 (329h16m46.235343077s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.432986750 (329h16m46.236451353s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.433950506 (329h16m46.237415105s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.434826879 (329h16m46.238291482s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.435543209 (329h16m46.23900781s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.436371462 (329h16m46.239836057s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.437175709 (329h16m46.240640311s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.438602053 (329h16m46.242066655s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.439766829 (329h16m46.243231432s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.440357301 (329h16m46.243821903s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.442091562 (329h16m46.245556166s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.443830317 (329h16m46.247294926s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.445663087 (329h16m46.249127688s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.447442216 (329h16m46.250906841s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:58.450984092 (329h16m46.254448692s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Uploading piece... Oct 7 06:46:59.005385383 (329h16m46.808849963s) [upload=289757]: Next returned nil segment Oct 7 06:46:59.005400448 (329h16m46.808865035s) [upload=289757]: Waiting for error group managing segments... Oct 7 06:46:59.143079058 (329h16m46.94654366s) [noise=true , address=77.90.31.11:4575 , piece_id=5ACLZRMYJHSV6ZMGLR4675MSGG4TUOGD7SRERGMWMCAAZJNJ3R2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.146273328 (329h16m46.949737924s) [noise=true , address=77.90.31.12:4611 , piece_id=Z5Y5VZZH6XGBNGINRZUFMLB6NWIXQPVNVIKT3XQHBR5ZF2ZNEN7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.148263295 (329h16m46.951727894s) [noise=true , address=45.91.116.6:4008 , piece_id=JEFNXHCSVCWEASZVH6U2ZDOMNA7HG3KYJH6IRXBXB3S5RDOFEYYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.160866418 (329h16m46.964331003s) [noise=true , address=45.91.116.14:4203 , piece_id=6QZRBB2UBTVQ44SXXXSY4VGZ4Z5NAC2YP3Y23I7QMTIZGPYJL62A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.179147670 (329h16m46.982612254s) [noise=true , address=77.90.31.10:4545 , piece_id=3AUXDPPNIWZXPOMG6FBM3FMRJATMNQ5XRNUV4GVPUSWD3ST7VI7A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.180704212 (329h16m46.984168799s) [noise=true , address=45.91.116.9:4288 , piece_id=SYTT4XFH3WFF2PIRXK3PSKD22JLSP5ZAKDOOOPZJ4KNVKA3KPC2Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.186070180 (329h16m46.989534776s) [noise=true , address=77.90.31.15:4703 , piece_id=TECFK6FLCRU3DO6X5VLCT57BAASVOX2HMCZPX7RRFBM6PL5DUZYA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.192209053 (329h16m46.995673651s) [noise=true , address=145.79.227.34:4908 , piece_id=BAU47JC4CC5IXJ6DSZKCYU75YM4FFEPIGAV7MDERWFVRO5JOX46Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.194447934 (329h16m46.997912524s) [noise=true , address=45.91.116.7:4039 , piece_id=MTUNUX5LX3RVMGEUPMKE44Y26JZVVTUZTIYNSU53H6NMMCF5UWVQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.202947254 (329h16m47.006411853s) [noise=true , address=145.79.227.23:4562 , piece_id=FNVYR7H5PNBMMTIMCCENWPVG6ABWNVBOLLPWVSLOMF7VZZ3OJJFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.212373254 (329h16m47.015837854s) [noise=true , address=145.79.227.41:4098 , piece_id=DXOYRODG6H5PWKSIULMSLHSGLZEILYBHZMLNYYVEIEYCNH6J7MXA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.213275884 (329h16m47.01674048s) [noise=true , address=145.79.227.42:4137 , piece_id=DRGTZJJSNXGLT4HN7P74ZT7NTFMFWZCBJHXWEM7A7YVJRU5DKGSQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:46:59.254435364 (329h16m47.057899947s) [noise=true , address=145.79.227.33:4898 , piece_id=2ZQHXVTJT4J6A7COOK3FUBE6TOXYQR23KFKDQ67R4OVH47Y6TWKA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: <nil> Oct 7 06:47:57.970656160 (329h17m45.774120739s) [noise=true , address=64.49.13.154:4935 , piece_id=K37WZY5XYGKQVTHW4V2R6NYDF3AWPWGFYKDWVA5ZUOANDLJ5PG4Q, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12GZpRx4CMfeRa5nUrXmBaTUxcRVfLkx6v4StbcVyWpVqeuZdfk, address:64.49.13.154:4935): protocol: read tcp 139.59.144.22:46854->64.49.13.154:4935: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970706330 (329h17m45.774170924s) [noise=true , address=64.49.13.138:4483 , piece_id=BPT3XOGZLJC2KXQEYESIOY7LIUHKNOAEYSVPOFQ6IBSWAGEIBVRQ, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: rpc: dial tcp 64.49.13.138:4483: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970776851 (329h17m45.774241441s) [noise=true , address=193.149.164.143:4698 , piece_id=PVMEO67YOXHXXBJDLGPOSPJJ7TTWIOBYVGC7K6B7A5SGFV4KI5SA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:12F4fJAGJM2ZPRt7vJH2cZmEbYeujmxcfi7aPZ2g9VPTNYBYRAi, address:193.149.164.143:4698): protocol: write tcp 139.59.144.22:39872->193.149.164.143:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.970862548 (329h17m45.774327127s) [noise=true , address=64.49.13.169:4714 , piece_id=SHWW62UMU2VRXIJWN5TYZJOZ52STTZJCE7UN6KWHSOMDSFHPVIFA, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1DsmdSnS2m7eL7TKCoyegaYK46jVo96qXPMxQWSaSCdZ7NbdKm, address:64.49.13.169:4714): protocol: read tcp 139.59.144.22:54486->64.49.13.169:4714: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:57.971032878 (329h17m45.774497471s) [noise=true , address=64.49.13.27:4963 , piece_id=G4EGVACWZZPCR6EABRZH3FGE7TWHUTNIQSKIA3FNAS3HBF2PQR6A, seg_pos={0 0}, upload=289757]: Done uploading piece. err: ecclient: upload failed (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq, address:64.49.13.27:4963): protocol: read tcp 139.59.144.22:57380->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:47:57.971071221 (329h17m45.774535815s) [seg_pos={0 0}, upload=289757]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 06:47:58.069707321 (329h17m45.873171915s) [upload=289757]: Done uploading segments... Oct 7 06:47:58.069739536 (329h17m45.873204132s) [upload=289757]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 2.281452384s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)