Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................X..........X...X.XX..X............] duration: 2.408155931s FAIL (2025-09-22 05:26:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.408189887s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 2.530678645s FAIL (2025-09-22 05:26:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.530717031s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.58839954s FAIL (2025-09-22 05:26:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.588451491s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X.XX......X....X..............X....X...XX...XX....XX......] duration: 2.789201597s FAIL (2025-09-22 05:25:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.789216224s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................X..........X...X.XX..X...........] duration: 2.366566005s FAIL (2025-09-22 05:25:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.366603386s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X.XX......X....X..............X....X...XX...XX....XX.....] duration: 2.781546797s FAIL (2025-09-22 05:24:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.781573726s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................X..........X...X.XX..X..........] duration: 2.630945629s FAIL (2025-09-22 05:24:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.630981157s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.503466356s FAIL (2025-09-22 05:24:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.503500749s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.720391121s FAIL (2025-09-22 05:24:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.72045132s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [....X.XX......X....X..............X....X...XX...XX....XX....] duration: 2.625345837s FAIL (2025-09-22 05:24:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.625376621s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.517404753s FAIL (2025-09-22 05:24:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.517438545s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.....X.XX......X....X..............X....X...XX...XX....XX...] duration: 2.927258632s FAIL (2025-09-22 05:23:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.927296428s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................X..........X...X.XX..X.........] duration: 2.630586272s FAIL (2025-09-22 05:23:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.630619147s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.821624752s FAIL (2025-09-22 05:23:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.821717816s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [......X.XX......X....X..............X....X...XX...XX....XX..] duration: 2.935956882s FAIL (2025-09-22 05:22:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.936001756s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................X..........X...X.XX..X........] duration: 2.430093406s FAIL (2025-09-22 05:22:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.430122669s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.57315577s FAIL (2025-09-22 05:22:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.573257524s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X..............................X..........X...X.XX..X.......] duration: 2.576622416s FAIL (2025-09-22 05:22:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.576656683s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......X.XX......X....X..............X....X...XX...XX....XX.] duration: 2.705855756s FAIL (2025-09-22 05:22:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.70595098s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [........X.XX......X....X..............X....X...XX...XX....XX] duration: 30.001058121s FAIL (2025-09-22 05:21:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001082337s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.417728691s FAIL (2025-09-22 05:21:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.417760234s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X..............................X..........X...X.XX..X......] duration: 2.437004341s FAIL (2025-09-22 05:21:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.437121589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.741656168s FAIL (2025-09-22 05:21:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.741695371s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X........X.XX......X....X..............X....X...XX...XX....X] duration: 30.001084321s FAIL (2025-09-22 05:21:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001097318s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.847518669s FAIL (2025-09-22 05:20:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.847561278s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.X..............................X..........X...X.XX..X.....] duration: 2.458368801s FAIL (2025-09-22 05:20:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.458399945s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X.X..............................X..........X...X.XX..X....] duration: 2.516991487s FAIL (2025-09-22 05:20:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.517092369s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.X........X.XX......X....X..............X....X...XX...XX....] duration: 2.784672304s FAIL (2025-09-22 05:19:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.784771224s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X.X..............................X..........X...X.XX..X...] duration: 2.406584389s FAIL (2025-09-22 05:19:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.406626646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...X.X..............................X..........X...X.XX..X..] duration: 2.32922128s FAIL (2025-09-22 05:19:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.329271328s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.577600076s FAIL (2025-09-22 05:19:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.577634318s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X.X........X.XX......X....X..............X....X...XX...XX...] duration: 3.273333641s FAIL (2025-09-22 05:19:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.27344739s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....X.X..............................X..........X...X.XX..X.] duration: 2.700407308s FAIL (2025-09-22 05:18:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.700437699s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X.X........X.XX......X....X..............X....X...XX...XX..] duration: 2.686175565s FAIL (2025-09-22 05:18:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.686208516s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.....X.X..............................X..........X...X.XX..X] duration: 30.116452347s FAIL (2025-09-22 05:18:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:17:59.679315241 (567h10m20.171747669s) [upload=498936]: Starting upload Sep 22 05:17:59.679411334 (567h10m20.171843786s) [upload=498936]: Done starting upload Sep 22 05:17:59.679716319 (567h10m20.172148755s) [upload=498936]: Uploading segments... Sep 22 05:17:59.679829213 (567h10m20.172261663s) [upload=498936]: Got next segment. Inline: false Sep 22 05:17:59.679950899 (567h10m20.172383345s) [upload=498936]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:17:59.679962985 (567h10m20.172395637s) [upload=498936]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:17:59.800475798 (567h10m20.292908243s) [seg_pos={0 0}, upload=498936]: Begin upload segment... Sep 22 05:17:59.800546024 (567h10m20.292978468s) [seg_pos={0 0}, upload=498936]: Done begin upload segment. Sep 22 05:17:59.800769539 (567h10m20.293201981s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.800893277 (567h10m20.293325723s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.801326339 (567h10m20.293758788s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802455300 (567h10m20.294887749s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.802758175 (567h10m20.29519062s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803082368 (567h10m20.295514813s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803319641 (567h10m20.295752094s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803736751 (567h10m20.296169199s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.803920030 (567h10m20.29635248s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804269267 (567h10m20.296701713s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804375095 (567h10m20.296807545s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804824050 (567h10m20.297256506s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.804984620 (567h10m20.297417069s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805400276 (567h10m20.297832723s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.805867188 (567h10m20.298299641s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:17:59.806095441 (567h10m20.298527898s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Uploading piece... Sep 22 05:18:00.412209571 (567h10m20.904642008s) [upload=498936]: Next returned nil segment Sep 22 05:18:00.412229059 (567h10m20.904661494s) [upload=498936]: Waiting for error group managing segments... Sep 22 05:18:00.545283219 (567h10m21.037715658s) [noise=true , address=45.91.116.10:4083 , piece_id=OMOQR5SDEXFDFUVALXACOQXOJGJOZDZL4XDT5PDGAUTJ4XJEDZWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.553265448 (567h10m21.045697895s) [noise=true , address=77.90.31.21:4879 , piece_id=TYGGGJWIDM5M5JYA2SKDJEMQOCQHL6OME3CSFLY3HADHV4MYO3WQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.554195368 (567h10m21.046627813s) [noise=true , address=45.91.116.7:4011 , piece_id=6DNA4SQ7HGJKVESDCFYN3WSDSQ2SP6CNJK6GZ6QREF4XOVTH4KWA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.555727408 (567h10m21.04815984s) [noise=true , address=45.91.116.4:4939 , piece_id=VD6PWIP25RFEUTU7N5TEJ7IBPKCCRG2RCMYVHUMIXPZJSJIW276A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.566137295 (567h10m21.05856974s) [noise=true , address=45.91.116.9:4262 , piece_id=VNP6XCBTUSFNSMILQUKODJMKKLFVX32FIN6H7NHWXPDON4M6SP3A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.576185664 (567h10m21.068618112s) [noise=true , address=77.90.31.15:4697 , piece_id=X7JLBHU4ZHFVHTWH3RKRBCN7AYRHU6I7DWYW7FZ6FKZHIVKSMNQA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579252726 (567h10m21.071685161s) [noise=true , address=45.91.116.5:4299 , piece_id=HAIBFEWPH3G2ZBEOVC6QT7GL4BBL3VBBZK4WQYLSQCOUNSCRFCCA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.579550122 (567h10m21.071982554s) [noise=true , address=77.90.31.19:4843 , piece_id=RGCNJNM2KLFJZ5WGPVTFUQ6UI7XRD677XFC4FMKJO4GJXRCDFUBA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.580613889 (567h10m21.073046322s) [noise=true , address=45.91.116.14:4190 , piece_id=BFBS3SISJGDT4DTWOXPO74KVBSO4MOEAB4HBESTHKN3RDWWSXPOQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.583606750 (567h10m21.076039182s) [noise=true , address=45.91.116.4:4940 , piece_id=HKBSDLNKF34EDIIRZ3VQP6V4F3GKW6N2PKMJWAZQVAWZOCYBDSIQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.584291982 (567h10m21.076724415s) [noise=true , address=77.90.31.12:4592 , piece_id=J2KPGJK6HPACQ7PK56EOYI7XW3U2ATVW32ZYTKI7AZV5XVRHG5CA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:00.610029812 (567h10m21.102462242s) [noise=true , address=45.91.116.11:4119 , piece_id=DG2ZFOJ2TENUA6FQOMK7MVJJHGWNWPMYUN6ZQ2TMMKYWUYVQBHUQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: <nil> Sep 22 05:18:02.263907796 (567h10m22.756340226s) [noise=true , address=145.79.227.16:4909 , piece_id=QRRSD42WLSTXJA6MJIFQM7CC2RM7MOMJXXQYYFBX6EAALNDFMWVA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370137159 (567h10m49.862569603s) [noise=true , address=145.79.227.13:4802 , piece_id=YHOT5235WSCCSERI24VLPZSZTHW4PCNESKRJXHS4MKR7JZ4BD4VQ, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: rpc: dial tcp 145.79.227.13:4802: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370226393 (567h10m49.862658847s) [noise=true , address=145.79.227.6:4259 , piece_id=NWUDFUQDOA2ELC4RWKCLTUFS53ZLVEURNDBWQSHB76DVJ5YHXTKA, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370259958 (567h10m49.862692512s) [noise=true , address=145.79.227.18:4005 , piece_id=D6PS2JVMRDFEDTAB26SQ36UWPRYHYT2XEYXHUKDNAPQTSUI2DV2A, seg_pos={0 0}, upload=498936]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:18:29.370300577 (567h10m49.862733023s) [seg_pos={0 0}, upload=498936]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:18:29.484566976 (567h10m49.976999411s) [upload=498936]: Done uploading segments... Sep 22 05:18:29.484595671 (567h10m49.977028104s) [upload=498936]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.116479523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.549747683s FAIL (2025-09-22 05:18:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.549781744s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X.X........X.XX......X....X..............X....X...XX...XX.] duration: 2.820342186s FAIL (2025-09-22 05:18:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.820548266s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.567938471s FAIL (2025-09-22 05:17:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.567973377s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X.X........X.XX......X....X..............X....X...XX...XX] duration: 30.00091343s FAIL (2025-09-22 05:17:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000932294s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [......X.X..............................X..........X...X.XX..] duration: 2.549685144s FAIL (2025-09-22 05:17:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== Sep 22 05:16:14.833356667 (567h8m35.325789117s) [upload=498909]: Starting upload Sep 22 05:16:14.833443381 (567h8m35.325875836s) [upload=498909]: Done starting upload Sep 22 05:16:14.833533885 (567h8m35.325966321s) [upload=498909]: Uploading segments... Sep 22 05:16:14.833751427 (567h8m35.326183855s) [upload=498909]: Got next segment. Inline: false Sep 22 05:16:14.833832899 (567h8m35.326265351s) [upload=498909]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:16:14.833850682 (567h8m35.326283128s) [upload=498909]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:16:14.955752024 (567h8m35.448184477s) [seg_pos={0 0}, upload=498909]: Begin upload segment... Sep 22 05:16:14.955873186 (567h8m35.448305619s) [seg_pos={0 0}, upload=498909]: Done begin upload segment. Sep 22 05:16:14.955905296 (567h8m35.448337749s) [noise=true , address=145.79.227.6:4257 , piece_id=X6DOEUNS3ZB34LDD7L2P7EP745I7WHTEREWTAUV2RGNAB6RDQQMA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.955940411 (567h8m35.448372869s) [noise=true , address=45.91.116.11:4117 , piece_id=K2T5HPYFXAED2JY7W4PXCPRDRAMEKRZTYETDX5JTU3U3V6D6CSFA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956416669 (567h8m35.448849121s) [noise=true , address=145.79.227.17:4971 , piece_id=6QHRG645EDTICHTVSEQSW6VMADKZCTQVHU54PCMXM4ODCBFNAV7Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956505931 (567h8m35.448938381s) [noise=true , address=145.79.227.18:4007 , piece_id=L3GJNHGCFKZ3S6NUJMGIM2WOYJUCMPBKAZVUDNQPBYGOHWPFAZ6Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956829176 (567h8m35.449261627s) [noise=true , address=145.79.227.14:4840 , piece_id=G6W5JSJLY2ROH5KXYYQ4VM4DCGNSYYIRMSWEAW3N5SCKR6T3T3WQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956994427 (567h8m35.449426881s) [noise=true , address=77.90.31.11:4553 , piece_id=JRWKQBKJ4ACOHNQ4T3NSBYETQC5CHYAFP3U3CFO7QHFILAK4EK2Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957189231 (567h8m35.449621683s) [noise=true , address=45.91.116.14:4189 , piece_id=PAIBUNB5FPMGXUTVO77SUFEFSHQLKMFLHJFPSIXSEXG7ONFSX2OQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957463299 (567h8m35.449895752s) [noise=true , address=145.79.227.2:4114 , piece_id=SQDAXCOSIX2B4PJ46TXPUYWUEQIM6WJKL6PVAR2CM37724HDEOWQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957556412 (567h8m35.449988861s) [noise=true , address=77.90.31.12:4590 , piece_id=LNOHFBDVFB6S6U5WBOFQIG6W5YWQW7BUMEFPHQSGXUC23DCFK2SA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957950173 (567h8m35.450382617s) [noise=true , address=45.91.116.10:4081 , piece_id=QWQKNT26VSBKYUTEW4JWXSCP3TDSVYVCD7UCUCLE5LE6U7TP3CJQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958021257 (567h8m35.45045371s) [noise=true , address=77.90.31.12:4592 , piece_id=RCGSM4OK7OKCB57OOZYE7YLIIIYMIE645B7J52KWK462SS6W3UTQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958426538 (567h8m35.450858992s) [noise=true , address=145.79.227.11:4731 , piece_id=HSWUDCHQBCUGN5RSI76LA5ICUW7DDZMXLFBMZZX2A3WYGR66LD3A, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958433348 (567h8m35.450865801s) [noise=true , address=77.90.31.13:4628 , piece_id=HBQ36D6A2HBOVIEGONK7SZO2YZNZNQMK3WULFYWOEQODM7SHGK2A, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958870625 (567h8m35.451303073s) [noise=true , address=145.79.227.15:4873 , piece_id=6EYOC4SSOV72I4KSBFCP5GFKWMO5FU2JOCF6I7IPLDOCP3G4LXPA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.960129963 (567h8m35.452562416s) [noise=true , address=145.79.227.12:4765 , piece_id=WQATCAAM6SN72P7IWUQ3HYOUMN63SZYF3M7LHWFREEWIOGHJ7BUQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.963315968 (567h8m35.455748421s) [noise=true , address=77.90.31.15:4698 , piece_id=NOJYBPYW3W5Z7CI6PNN6MXHCMN4YXGICHRWX2PW5VDRCLLG6NGCA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:15.576038727 (567h8m36.068471164s) [upload=498909]: Next returned nil segment Sep 22 05:16:15.576060439 (567h8m36.068492869s) [upload=498909]: Waiting for error group managing segments... Sep 22 05:16:15.707264907 (567h8m36.199697342s) [noise=true , address=45.91.116.10:4081 , piece_id=QWQKNT26VSBKYUTEW4JWXSCP3TDSVYVCD7UCUCLE5LE6U7TP3CJQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.710902725 (567h8m36.203335162s) [noise=true , address=45.91.116.14:4189 , piece_id=PAIBUNB5FPMGXUTVO77SUFEFSHQLKMFLHJFPSIXSEXG7ONFSX2OQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.713202714 (567h8m36.205635163s) [noise=true , address=77.90.31.12:4590 , piece_id=LNOHFBDVFB6S6U5WBOFQIG6W5YWQW7BUMEFPHQSGXUC23DCFK2SA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.716774677 (567h8m36.209207105s) [noise=true , address=77.90.31.12:4592 , piece_id=RCGSM4OK7OKCB57OOZYE7YLIIIYMIE645B7J52KWK462SS6W3UTQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.719343117 (567h8m36.211775557s) [noise=true , address=45.91.116.11:4117 , piece_id=K2T5HPYFXAED2JY7W4PXCPRDRAMEKRZTYETDX5JTU3U3V6D6CSFA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.729031434 (567h8m36.221463863s) [noise=true , address=77.90.31.11:4553 , piece_id=JRWKQBKJ4ACOHNQ4T3NSBYETQC5CHYAFP3U3CFO7QHFILAK4EK2Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.740560788 (567h8m36.232993224s) [noise=true , address=77.90.31.15:4698 , piece_id=NOJYBPYW3W5Z7CI6PNN6MXHCMN4YXGICHRWX2PW5VDRCLLG6NGCA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.746265539 (567h8m36.23869797s) [noise=true , address=77.90.31.13:4628 , piece_id=HBQ36D6A2HBOVIEGONK7SZO2YZNZNQMK3WULFYWOEQODM7SHGK2A, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:44.514850309 (567h9m5.00728276s) [noise=true , address=145.79.227.6:4257 , piece_id=X6DOEUNS3ZB34LDD7L2P7EP745I7WHTEREWTAUV2RGNAB6RDQQMA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: context deadline exceeded; context deadline exceeded Sep 22 05:16:44.515329650 (567h9m5.007762104s) [noise=true , address=145.79.227.17:4971 , piece_id=6QHRG645EDTICHTVSEQSW6VMADKZCTQVHU54PCMXM4ODCBFNAV7Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515396336 (567h9m5.00782878s) [noise=true , address=145.79.227.18:4007 , piece_id=L3GJNHGCFKZ3S6NUJMGIM2WOYJUCMPBKAZVUDNQPBYGOHWPFAZ6Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515497411 (567h9m5.00792986s) [noise=true , address=145.79.227.14:4840 , piece_id=G6W5JSJLY2ROH5KXYYQ4VM4DCGNSYYIRMSWEAW3N5SCKR6T3T3WQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515572202 (567h9m5.008004648s) [noise=true , address=145.79.227.2:4114 , piece_id=SQDAXCOSIX2B4PJ46TXPUYWUEQIM6WJKL6PVAR2CM37724HDEOWQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515597528 (567h9m5.008029976s) [noise=true , address=145.79.227.12:4765 , piece_id=WQATCAAM6SN72P7IWUQ3HYOUMN63SZYF3M7LHWFREEWIOGHJ7BUQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515643776 (567h9m5.008076223s) [noise=true , address=145.79.227.11:4731 , piece_id=HSWUDCHQBCUGN5RSI76LA5ICUW7DDZMXLFBMZZX2A3WYGR66LD3A, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515658290 (567h9m5.008090738s) [noise=true , address=145.79.227.15:4873 , piece_id=6EYOC4SSOV72I4KSBFCP5GFKWMO5FU2JOCF6I7IPLDOCP3G4LXPA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515690161 (567h9m5.008122614s) [seg_pos={0 0}, upload=498909]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Sep 22 05:16:44.627709732 (567h9m5.120142174s) [upload=498909]: Done uploading segments... Sep 22 05:16:44.627765912 (567h9m5.120198353s) [upload=498909]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.549715803s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......X.X..............................X..........X...X.XX.] duration: 2.457912842s FAIL (2025-09-22 05:17:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== Sep 22 05:16:14.833356667 (567h8m35.325789117s) [upload=498909]: Starting upload Sep 22 05:16:14.833443381 (567h8m35.325875836s) [upload=498909]: Done starting upload Sep 22 05:16:14.833533885 (567h8m35.325966321s) [upload=498909]: Uploading segments... Sep 22 05:16:14.833751427 (567h8m35.326183855s) [upload=498909]: Got next segment. Inline: false Sep 22 05:16:14.833832899 (567h8m35.326265351s) [upload=498909]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:16:14.833850682 (567h8m35.326283128s) [upload=498909]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:16:14.955752024 (567h8m35.448184477s) [seg_pos={0 0}, upload=498909]: Begin upload segment... Sep 22 05:16:14.955873186 (567h8m35.448305619s) [seg_pos={0 0}, upload=498909]: Done begin upload segment. Sep 22 05:16:14.955905296 (567h8m35.448337749s) [noise=true , address=145.79.227.6:4257 , piece_id=X6DOEUNS3ZB34LDD7L2P7EP745I7WHTEREWTAUV2RGNAB6RDQQMA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.955940411 (567h8m35.448372869s) [noise=true , address=45.91.116.11:4117 , piece_id=K2T5HPYFXAED2JY7W4PXCPRDRAMEKRZTYETDX5JTU3U3V6D6CSFA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956416669 (567h8m35.448849121s) [noise=true , address=145.79.227.17:4971 , piece_id=6QHRG645EDTICHTVSEQSW6VMADKZCTQVHU54PCMXM4ODCBFNAV7Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956505931 (567h8m35.448938381s) [noise=true , address=145.79.227.18:4007 , piece_id=L3GJNHGCFKZ3S6NUJMGIM2WOYJUCMPBKAZVUDNQPBYGOHWPFAZ6Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956829176 (567h8m35.449261627s) [noise=true , address=145.79.227.14:4840 , piece_id=G6W5JSJLY2ROH5KXYYQ4VM4DCGNSYYIRMSWEAW3N5SCKR6T3T3WQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956994427 (567h8m35.449426881s) [noise=true , address=77.90.31.11:4553 , piece_id=JRWKQBKJ4ACOHNQ4T3NSBYETQC5CHYAFP3U3CFO7QHFILAK4EK2Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957189231 (567h8m35.449621683s) [noise=true , address=45.91.116.14:4189 , piece_id=PAIBUNB5FPMGXUTVO77SUFEFSHQLKMFLHJFPSIXSEXG7ONFSX2OQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957463299 (567h8m35.449895752s) [noise=true , address=145.79.227.2:4114 , piece_id=SQDAXCOSIX2B4PJ46TXPUYWUEQIM6WJKL6PVAR2CM37724HDEOWQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957556412 (567h8m35.449988861s) [noise=true , address=77.90.31.12:4590 , piece_id=LNOHFBDVFB6S6U5WBOFQIG6W5YWQW7BUMEFPHQSGXUC23DCFK2SA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957950173 (567h8m35.450382617s) [noise=true , address=45.91.116.10:4081 , piece_id=QWQKNT26VSBKYUTEW4JWXSCP3TDSVYVCD7UCUCLE5LE6U7TP3CJQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958021257 (567h8m35.45045371s) [noise=true , address=77.90.31.12:4592 , piece_id=RCGSM4OK7OKCB57OOZYE7YLIIIYMIE645B7J52KWK462SS6W3UTQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958426538 (567h8m35.450858992s) [noise=true , address=145.79.227.11:4731 , piece_id=HSWUDCHQBCUGN5RSI76LA5ICUW7DDZMXLFBMZZX2A3WYGR66LD3A, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958433348 (567h8m35.450865801s) [noise=true , address=77.90.31.13:4628 , piece_id=HBQ36D6A2HBOVIEGONK7SZO2YZNZNQMK3WULFYWOEQODM7SHGK2A, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958870625 (567h8m35.451303073s) [noise=true , address=145.79.227.15:4873 , piece_id=6EYOC4SSOV72I4KSBFCP5GFKWMO5FU2JOCF6I7IPLDOCP3G4LXPA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.960129963 (567h8m35.452562416s) [noise=true , address=145.79.227.12:4765 , piece_id=WQATCAAM6SN72P7IWUQ3HYOUMN63SZYF3M7LHWFREEWIOGHJ7BUQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.963315968 (567h8m35.455748421s) [noise=true , address=77.90.31.15:4698 , piece_id=NOJYBPYW3W5Z7CI6PNN6MXHCMN4YXGICHRWX2PW5VDRCLLG6NGCA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:15.576038727 (567h8m36.068471164s) [upload=498909]: Next returned nil segment Sep 22 05:16:15.576060439 (567h8m36.068492869s) [upload=498909]: Waiting for error group managing segments... Sep 22 05:16:15.707264907 (567h8m36.199697342s) [noise=true , address=45.91.116.10:4081 , piece_id=QWQKNT26VSBKYUTEW4JWXSCP3TDSVYVCD7UCUCLE5LE6U7TP3CJQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.710902725 (567h8m36.203335162s) [noise=true , address=45.91.116.14:4189 , piece_id=PAIBUNB5FPMGXUTVO77SUFEFSHQLKMFLHJFPSIXSEXG7ONFSX2OQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.713202714 (567h8m36.205635163s) [noise=true , address=77.90.31.12:4590 , piece_id=LNOHFBDVFB6S6U5WBOFQIG6W5YWQW7BUMEFPHQSGXUC23DCFK2SA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.716774677 (567h8m36.209207105s) [noise=true , address=77.90.31.12:4592 , piece_id=RCGSM4OK7OKCB57OOZYE7YLIIIYMIE645B7J52KWK462SS6W3UTQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.719343117 (567h8m36.211775557s) [noise=true , address=45.91.116.11:4117 , piece_id=K2T5HPYFXAED2JY7W4PXCPRDRAMEKRZTYETDX5JTU3U3V6D6CSFA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.729031434 (567h8m36.221463863s) [noise=true , address=77.90.31.11:4553 , piece_id=JRWKQBKJ4ACOHNQ4T3NSBYETQC5CHYAFP3U3CFO7QHFILAK4EK2Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.740560788 (567h8m36.232993224s) [noise=true , address=77.90.31.15:4698 , piece_id=NOJYBPYW3W5Z7CI6PNN6MXHCMN4YXGICHRWX2PW5VDRCLLG6NGCA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.746265539 (567h8m36.23869797s) [noise=true , address=77.90.31.13:4628 , piece_id=HBQ36D6A2HBOVIEGONK7SZO2YZNZNQMK3WULFYWOEQODM7SHGK2A, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:44.514850309 (567h9m5.00728276s) [noise=true , address=145.79.227.6:4257 , piece_id=X6DOEUNS3ZB34LDD7L2P7EP745I7WHTEREWTAUV2RGNAB6RDQQMA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: context deadline exceeded; context deadline exceeded Sep 22 05:16:44.515329650 (567h9m5.007762104s) [noise=true , address=145.79.227.17:4971 , piece_id=6QHRG645EDTICHTVSEQSW6VMADKZCTQVHU54PCMXM4ODCBFNAV7Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515396336 (567h9m5.00782878s) [noise=true , address=145.79.227.18:4007 , piece_id=L3GJNHGCFKZ3S6NUJMGIM2WOYJUCMPBKAZVUDNQPBYGOHWPFAZ6Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515497411 (567h9m5.00792986s) [noise=true , address=145.79.227.14:4840 , piece_id=G6W5JSJLY2ROH5KXYYQ4VM4DCGNSYYIRMSWEAW3N5SCKR6T3T3WQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515572202 (567h9m5.008004648s) [noise=true , address=145.79.227.2:4114 , piece_id=SQDAXCOSIX2B4PJ46TXPUYWUEQIM6WJKL6PVAR2CM37724HDEOWQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515597528 (567h9m5.008029976s) [noise=true , address=145.79.227.12:4765 , piece_id=WQATCAAM6SN72P7IWUQ3HYOUMN63SZYF3M7LHWFREEWIOGHJ7BUQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515643776 (567h9m5.008076223s) [noise=true , address=145.79.227.11:4731 , piece_id=HSWUDCHQBCUGN5RSI76LA5ICUW7DDZMXLFBMZZX2A3WYGR66LD3A, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515658290 (567h9m5.008090738s) [noise=true , address=145.79.227.15:4873 , piece_id=6EYOC4SSOV72I4KSBFCP5GFKWMO5FU2JOCF6I7IPLDOCP3G4LXPA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515690161 (567h9m5.008122614s) [seg_pos={0 0}, upload=498909]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Sep 22 05:16:44.627709732 (567h9m5.120142174s) [upload=498909]: Done uploading segments... Sep 22 05:16:44.627765912 (567h9m5.120198353s) [upload=498909]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 2.45794639s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.796229476s FAIL (2025-09-22 05:17:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.796291029s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [........X.X..............................X..........X...X.XX] duration: 30.114887435s FAIL (2025-09-22 05:16:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== Sep 22 05:16:14.833356667 (567h8m35.325789117s) [upload=498909]: Starting upload Sep 22 05:16:14.833443381 (567h8m35.325875836s) [upload=498909]: Done starting upload Sep 22 05:16:14.833533885 (567h8m35.325966321s) [upload=498909]: Uploading segments... Sep 22 05:16:14.833751427 (567h8m35.326183855s) [upload=498909]: Got next segment. Inline: false Sep 22 05:16:14.833832899 (567h8m35.326265351s) [upload=498909]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:16:14.833850682 (567h8m35.326283128s) [upload=498909]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:16:14.955752024 (567h8m35.448184477s) [seg_pos={0 0}, upload=498909]: Begin upload segment... Sep 22 05:16:14.955873186 (567h8m35.448305619s) [seg_pos={0 0}, upload=498909]: Done begin upload segment. Sep 22 05:16:14.955905296 (567h8m35.448337749s) [noise=true , address=145.79.227.6:4257 , piece_id=X6DOEUNS3ZB34LDD7L2P7EP745I7WHTEREWTAUV2RGNAB6RDQQMA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.955940411 (567h8m35.448372869s) [noise=true , address=45.91.116.11:4117 , piece_id=K2T5HPYFXAED2JY7W4PXCPRDRAMEKRZTYETDX5JTU3U3V6D6CSFA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956416669 (567h8m35.448849121s) [noise=true , address=145.79.227.17:4971 , piece_id=6QHRG645EDTICHTVSEQSW6VMADKZCTQVHU54PCMXM4ODCBFNAV7Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956505931 (567h8m35.448938381s) [noise=true , address=145.79.227.18:4007 , piece_id=L3GJNHGCFKZ3S6NUJMGIM2WOYJUCMPBKAZVUDNQPBYGOHWPFAZ6Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956829176 (567h8m35.449261627s) [noise=true , address=145.79.227.14:4840 , piece_id=G6W5JSJLY2ROH5KXYYQ4VM4DCGNSYYIRMSWEAW3N5SCKR6T3T3WQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.956994427 (567h8m35.449426881s) [noise=true , address=77.90.31.11:4553 , piece_id=JRWKQBKJ4ACOHNQ4T3NSBYETQC5CHYAFP3U3CFO7QHFILAK4EK2Q, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957189231 (567h8m35.449621683s) [noise=true , address=45.91.116.14:4189 , piece_id=PAIBUNB5FPMGXUTVO77SUFEFSHQLKMFLHJFPSIXSEXG7ONFSX2OQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957463299 (567h8m35.449895752s) [noise=true , address=145.79.227.2:4114 , piece_id=SQDAXCOSIX2B4PJ46TXPUYWUEQIM6WJKL6PVAR2CM37724HDEOWQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957556412 (567h8m35.449988861s) [noise=true , address=77.90.31.12:4590 , piece_id=LNOHFBDVFB6S6U5WBOFQIG6W5YWQW7BUMEFPHQSGXUC23DCFK2SA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.957950173 (567h8m35.450382617s) [noise=true , address=45.91.116.10:4081 , piece_id=QWQKNT26VSBKYUTEW4JWXSCP3TDSVYVCD7UCUCLE5LE6U7TP3CJQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958021257 (567h8m35.45045371s) [noise=true , address=77.90.31.12:4592 , piece_id=RCGSM4OK7OKCB57OOZYE7YLIIIYMIE645B7J52KWK462SS6W3UTQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958426538 (567h8m35.450858992s) [noise=true , address=145.79.227.11:4731 , piece_id=HSWUDCHQBCUGN5RSI76LA5ICUW7DDZMXLFBMZZX2A3WYGR66LD3A, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958433348 (567h8m35.450865801s) [noise=true , address=77.90.31.13:4628 , piece_id=HBQ36D6A2HBOVIEGONK7SZO2YZNZNQMK3WULFYWOEQODM7SHGK2A, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.958870625 (567h8m35.451303073s) [noise=true , address=145.79.227.15:4873 , piece_id=6EYOC4SSOV72I4KSBFCP5GFKWMO5FU2JOCF6I7IPLDOCP3G4LXPA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.960129963 (567h8m35.452562416s) [noise=true , address=145.79.227.12:4765 , piece_id=WQATCAAM6SN72P7IWUQ3HYOUMN63SZYF3M7LHWFREEWIOGHJ7BUQ, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:14.963315968 (567h8m35.455748421s) [noise=true , address=77.90.31.15:4698 , piece_id=NOJYBPYW3W5Z7CI6PNN6MXHCMN4YXGICHRWX2PW5VDRCLLG6NGCA, seg_pos={0 0}, upload=498909]: Uploading piece... Sep 22 05:16:15.576038727 (567h8m36.068471164s) [upload=498909]: Next returned nil segment Sep 22 05:16:15.576060439 (567h8m36.068492869s) [upload=498909]: Waiting for error group managing segments... Sep 22 05:16:15.707264907 (567h8m36.199697342s) [noise=true , address=45.91.116.10:4081 , piece_id=QWQKNT26VSBKYUTEW4JWXSCP3TDSVYVCD7UCUCLE5LE6U7TP3CJQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.710902725 (567h8m36.203335162s) [noise=true , address=45.91.116.14:4189 , piece_id=PAIBUNB5FPMGXUTVO77SUFEFSHQLKMFLHJFPSIXSEXG7ONFSX2OQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.713202714 (567h8m36.205635163s) [noise=true , address=77.90.31.12:4590 , piece_id=LNOHFBDVFB6S6U5WBOFQIG6W5YWQW7BUMEFPHQSGXUC23DCFK2SA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.716774677 (567h8m36.209207105s) [noise=true , address=77.90.31.12:4592 , piece_id=RCGSM4OK7OKCB57OOZYE7YLIIIYMIE645B7J52KWK462SS6W3UTQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.719343117 (567h8m36.211775557s) [noise=true , address=45.91.116.11:4117 , piece_id=K2T5HPYFXAED2JY7W4PXCPRDRAMEKRZTYETDX5JTU3U3V6D6CSFA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.729031434 (567h8m36.221463863s) [noise=true , address=77.90.31.11:4553 , piece_id=JRWKQBKJ4ACOHNQ4T3NSBYETQC5CHYAFP3U3CFO7QHFILAK4EK2Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.740560788 (567h8m36.232993224s) [noise=true , address=77.90.31.15:4698 , piece_id=NOJYBPYW3W5Z7CI6PNN6MXHCMN4YXGICHRWX2PW5VDRCLLG6NGCA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:15.746265539 (567h8m36.23869797s) [noise=true , address=77.90.31.13:4628 , piece_id=HBQ36D6A2HBOVIEGONK7SZO2YZNZNQMK3WULFYWOEQODM7SHGK2A, seg_pos={0 0}, upload=498909]: Done uploading piece. err: <nil> Sep 22 05:16:44.514850309 (567h9m5.00728276s) [noise=true , address=145.79.227.6:4257 , piece_id=X6DOEUNS3ZB34LDD7L2P7EP745I7WHTEREWTAUV2RGNAB6RDQQMA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: context deadline exceeded; context deadline exceeded Sep 22 05:16:44.515329650 (567h9m5.007762104s) [noise=true , address=145.79.227.17:4971 , piece_id=6QHRG645EDTICHTVSEQSW6VMADKZCTQVHU54PCMXM4ODCBFNAV7Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: rpc: dial tcp 145.79.227.17:4971: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515396336 (567h9m5.00782878s) [noise=true , address=145.79.227.18:4007 , piece_id=L3GJNHGCFKZ3S6NUJMGIM2WOYJUCMPBKAZVUDNQPBYGOHWPFAZ6Q, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: rpc: dial tcp 145.79.227.18:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515497411 (567h9m5.00792986s) [noise=true , address=145.79.227.14:4840 , piece_id=G6W5JSJLY2ROH5KXYYQ4VM4DCGNSYYIRMSWEAW3N5SCKR6T3T3WQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515572202 (567h9m5.008004648s) [noise=true , address=145.79.227.2:4114 , piece_id=SQDAXCOSIX2B4PJ46TXPUYWUEQIM6WJKL6PVAR2CM37724HDEOWQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: rpc: dial tcp 145.79.227.2:4114: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515597528 (567h9m5.008029976s) [noise=true , address=145.79.227.12:4765 , piece_id=WQATCAAM6SN72P7IWUQ3HYOUMN63SZYF3M7LHWFREEWIOGHJ7BUQ, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515643776 (567h9m5.008076223s) [noise=true , address=145.79.227.11:4731 , piece_id=HSWUDCHQBCUGN5RSI76LA5ICUW7DDZMXLFBMZZX2A3WYGR66LD3A, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515658290 (567h9m5.008090738s) [noise=true , address=145.79.227.15:4873 , piece_id=6EYOC4SSOV72I4KSBFCP5GFKWMO5FU2JOCF6I7IPLDOCP3G4LXPA, seg_pos={0 0}, upload=498909]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:16:44.515690161 (567h9m5.008122614s) [seg_pos={0 0}, upload=498909]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Sep 22 05:16:44.627709732 (567h9m5.120142174s) [upload=498909]: Done uploading segments... Sep 22 05:16:44.627765912 (567h9m5.120198353s) [upload=498909]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.114916459s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....X.X........X.XX......X....X..............X....X...XX...X] duration: 30.000788003s FAIL (2025-09-22 05:16:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000826422s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.676687517s FAIL (2025-09-22 05:16:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.676718074s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X........X.X..............................X..........X...X.X] duration: 30.108880026s FAIL (2025-09-22 05:15:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:14:59.812033716 (567h7m20.304466147s) [upload=498892]: Starting upload Sep 22 05:14:59.812109400 (567h7m20.304541842s) [upload=498892]: Done starting upload Sep 22 05:14:59.812210092 (567h7m20.304642542s) [upload=498892]: Uploading segments... Sep 22 05:14:59.812419743 (567h7m20.304852176s) [upload=498892]: Got next segment. Inline: false Sep 22 05:14:59.812478957 (567h7m20.30491139s) [upload=498892]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:14:59.812484587 (567h7m20.30491702s) [upload=498892]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:14:59.925136815 (567h7m20.41756926s) [seg_pos={0 0}, upload=498892]: Begin upload segment... Sep 22 05:14:59.925192912 (567h7m20.417625354s) [seg_pos={0 0}, upload=498892]: Done begin upload segment. Sep 22 05:14:59.925231847 (567h7m20.417664298s) [noise=true , address=77.90.31.11:4554 , piece_id=GXLVTMWSYNJM7AJPXYKWK36NNAZYHWWCAQU5LMCBGUTAWRNMN5SQ, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.925626053 (567h7m20.418058502s) [noise=true , address=45.91.116.14:4189 , piece_id=PMOHCC63AV7TEWNU2EO6TNVUH4MGWGLB7JPTXHCCGUPLYITDCNQA, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.925753106 (567h7m20.418185548s) [noise=true , address=77.90.31.13:4626 , piece_id=3EYTDCSL5MMIZBZG4CB247BCDFPSMNU7KDZVNCSMMAYM4FQRKU7Q, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.926142638 (567h7m20.418575082s) [noise=true , address=145.79.227.13:4801 , piece_id=ARJDDNOIQF34YPZVJHUHBLEDV6ECIPB5HUWAKT7PPDJYGVMOJE3A, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.927149574 (567h7m20.419582022s) [noise=true , address=145.79.227.5:4222 , piece_id=LJFXE7H3A6V4UX4WVMRYNJAXP6K4KHJFP5YTW3O5FYAILKSX3N6A, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.928216956 (567h7m20.420649407s) [noise=true , address=145.79.227.6:4257 , piece_id=GYTWXPZPFTMVWW6EH2E6MUKPYA7YGGJMRY4PY5IOW4G6Y6GZQ7GA, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.928251987 (567h7m20.420684431s) [noise=true , address=77.90.31.17:4770 , piece_id=IIIZ6XTOCLGJIXWQ7ZCVOE4Y45NWQ4INXUNICA5TEEMZAJ4Q2DDQ, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.928914873 (567h7m20.42134732s) [noise=true , address=45.91.116.5:4297 , piece_id=I7JXNWI6A2P6OJ3MEWIP2DL3WFAX5DOLHEOXD336LA4KMJ7ORO5A, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.929458002 (567h7m20.42189045s) [noise=true , address=77.90.31.21:4877 , piece_id=7YOEPMIPIK25QCPMRCW6SBQAPRIILLBLN7IR7PRPPUPDKFYGZOEQ, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.929881874 (567h7m20.422314322s) [noise=true , address=77.90.31.12:4592 , piece_id=S53LFCOUEYLMFIOQ4ZC5LKXMA64AHCETYMBHXAEQ54GOOWAYSHGQ, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.930329319 (567h7m20.422761772s) [noise=true , address=45.91.116.8:4048 , piece_id=LYS52AAU54QVTL75VIPLSQVKZYB2BIVHJLFGV6ST67QNTUXCR77Q, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.930673952 (567h7m20.423106398s) [noise=true , address=45.91.116.12:4225 , piece_id=PJ7DBEGK7MRCQL66LUA3K4OVG32X67FK6AU5NOKANG7ZDQWFHMXQ, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.930950639 (567h7m20.423383081s) [noise=true , address=145.79.227.3:4150 , piece_id=N5UZWCYZKVLVNZTETFJDPP5Y2KD5F74O3LZOXTGCOQKFVAODUTIA, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.931817671 (567h7m20.424250115s) [noise=true , address=45.91.116.9:4261 , piece_id=7TQ3WBRZ3ZQULWV3JBZP7SDLZNIGZMYHTAKISBISXXN6CGWQD55A, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.932165475 (567h7m20.424597918s) [noise=true , address=45.91.116.10:4083 , piece_id=MISW7QPVOCJKW6GNJAAF4DAKJYGMVQEVCE3VOJWIC6Q2E34LQOQQ, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:14:59.933688189 (567h7m20.426120634s) [noise=true , address=77.90.31.18:4805 , piece_id=33OVTVSNXB7IEPPW5EZTYFHHKPXK6OF4CIWWAUPEPBVCWI4VYUGA, seg_pos={0 0}, upload=498892]: Uploading piece... Sep 22 05:15:00.556063068 (567h7m21.048495514s) [upload=498892]: Next returned nil segment Sep 22 05:15:00.556106309 (567h7m21.04853876s) [upload=498892]: Waiting for error group managing segments... Sep 22 05:15:00.686429774 (567h7m21.178862208s) [noise=true , address=45.91.116.5:4297 , piece_id=I7JXNWI6A2P6OJ3MEWIP2DL3WFAX5DOLHEOXD336LA4KMJ7ORO5A, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.692223782 (567h7m21.184656217s) [noise=true , address=45.91.116.12:4225 , piece_id=PJ7DBEGK7MRCQL66LUA3K4OVG32X67FK6AU5NOKANG7ZDQWFHMXQ, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.693044278 (567h7m21.185476712s) [noise=true , address=77.90.31.21:4877 , piece_id=7YOEPMIPIK25QCPMRCW6SBQAPRIILLBLN7IR7PRPPUPDKFYGZOEQ, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.695957135 (567h7m21.188389578s) [noise=true , address=45.91.116.10:4083 , piece_id=MISW7QPVOCJKW6GNJAAF4DAKJYGMVQEVCE3VOJWIC6Q2E34LQOQQ, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.696132180 (567h7m21.188564617s) [noise=true , address=45.91.116.14:4189 , piece_id=PMOHCC63AV7TEWNU2EO6TNVUH4MGWGLB7JPTXHCCGUPLYITDCNQA, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.698625677 (567h7m21.191058117s) [noise=true , address=77.90.31.13:4626 , piece_id=3EYTDCSL5MMIZBZG4CB247BCDFPSMNU7KDZVNCSMMAYM4FQRKU7Q, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.699609983 (567h7m21.192042422s) [noise=true , address=77.90.31.18:4805 , piece_id=33OVTVSNXB7IEPPW5EZTYFHHKPXK6OF4CIWWAUPEPBVCWI4VYUGA, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.700455627 (567h7m21.192888066s) [noise=true , address=45.91.116.9:4261 , piece_id=7TQ3WBRZ3ZQULWV3JBZP7SDLZNIGZMYHTAKISBISXXN6CGWQD55A, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.706377293 (567h7m21.198809737s) [noise=true , address=77.90.31.12:4592 , piece_id=S53LFCOUEYLMFIOQ4ZC5LKXMA64AHCETYMBHXAEQ54GOOWAYSHGQ, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.706855605 (567h7m21.199288043s) [noise=true , address=77.90.31.17:4770 , piece_id=IIIZ6XTOCLGJIXWQ7ZCVOE4Y45NWQ4INXUNICA5TEEMZAJ4Q2DDQ, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:00.710839850 (567h7m21.203272282s) [noise=true , address=45.91.116.8:4048 , piece_id=LYS52AAU54QVTL75VIPLSQVKZYB2BIVHJLFGV6ST67QNTUXCR77Q, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:01.649238534 (567h7m22.141670989s) [noise=true , address=145.79.227.3:4150 , piece_id=N5UZWCYZKVLVNZTETFJDPP5Y2KD5F74O3LZOXTGCOQKFVAODUTIA, seg_pos={0 0}, upload=498892]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: rpc: dial tcp 145.79.227.3:4150: connect: no route to host; conn closed: op Sep 22 05:15:01.714299807 (567h7m22.206732253s) [noise=true , address=77.90.31.11:4554 , piece_id=GXLVTMWSYNJM7AJPXYKWK36NNAZYHWWCAQU5LMCBGUTAWRNMN5SQ, seg_pos={0 0}, upload=498892]: Done uploading piece. err: <nil> Sep 22 05:15:03.728663313 (567h7m24.221095747s) [noise=true , address=145.79.227.5:4222 , piece_id=LJFXE7H3A6V4UX4WVMRYNJAXP6K4KHJFP5YTW3O5FYAILKSX3N6A, seg_pos={0 0}, upload=498892]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: connect: no route to host; conn closed: op Sep 22 05:15:29.529814145 (567h7m50.022246588s) [noise=true , address=145.79.227.6:4257 , piece_id=GYTWXPZPFTMVWW6EH2E6MUKPYA7YGGJMRY4PY5IOW4G6Y6GZQ7GA, seg_pos={0 0}, upload=498892]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: rpc: dial tcp 145.79.227.6:4257: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:15:29.529888990 (567h7m50.022321424s) [noise=true , address=145.79.227.13:4801 , piece_id=ARJDDNOIQF34YPZVJHUHBLEDV6ECIPB5HUWAKT7PPDJYGVMOJE3A, seg_pos={0 0}, upload=498892]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op Sep 22 05:15:29.529907466 (567h7m50.022339898s) [seg_pos={0 0}, upload=498892]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:15:29.637155222 (567h7m50.129587653s) [upload=498892]: Done uploading segments... Sep 22 05:15:29.637195624 (567h7m50.129628055s) [upload=498892]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.108934593s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....X.X........X.XX......X....X..............X....X...XX...] duration: 2.896629546s FAIL (2025-09-22 05:14:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.896657803s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.482588924s FAIL (2025-09-22 05:14:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.482615348s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X........X.X..............................X..........X...X.] duration: 2.58273643s FAIL (2025-09-22 05:14:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:12:52.523477769 (567h5m13.015910235s) [upload=498863]: Starting upload Sep 22 05:12:52.523581725 (567h5m13.016014175s) [upload=498863]: Done starting upload Sep 22 05:12:52.523707123 (567h5m13.016139556s) [upload=498863]: Uploading segments... Sep 22 05:12:52.523991347 (567h5m13.01642381s) [upload=498863]: Got next segment. Inline: false Sep 22 05:12:52.524075179 (567h5m13.01650763s) [upload=498863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:12:52.524094821 (567h5m13.016527282s) [upload=498863]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:12:52.636839862 (567h5m13.129272316s) [seg_pos={0 0}, upload=498863]: Begin upload segment... Sep 22 05:12:52.636918319 (567h5m13.129350759s) [seg_pos={0 0}, upload=498863]: Done begin upload segment. Sep 22 05:12:52.636952839 (567h5m13.129385281s) [noise=true , address=45.91.116.11:4117 , piece_id=63J6DYPINT6TU75XTD4S7EZ7SVNVKKPADERJ3K4463QHCRO7GPPQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.636983865 (567h5m13.129416314s) [noise=true , address=145.79.227.16:4909 , piece_id=Z5WGVCFFQ6GUXEIBQXKOZQDZM3E5RRPXXE6BLSBA7LVODAWWVLYA, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.637311966 (567h5m13.129744395s) [noise=true , address=45.91.116.14:4189 , piece_id=JLEHWHBNZTKTYM4OAWCY44TMZYLNEG5A7KGXMCABB5Y677VPK42Q, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.637521319 (567h5m13.129953771s) [noise=true , address=77.90.31.14:4663 , piece_id=KLHWWAPPYZXKDJJ7U3PGGQ4PGBFFP35TLTQOCBRKXIOEDR3LCMZQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.642153061 (567h5m13.134585513s) [noise=true , address=77.90.31.18:4805 , piece_id=JHBXQ6UJN3GN5YHF6ODWWKCY7DDNNDEL3HCXBAJHAJVPYBJYHYGQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.642630539 (567h5m13.135062991s) [noise=true , address=45.91.116.7:4009 , piece_id=HPJJTRLI43FLOMJCVQ4R2YT6F6WQBWTCZV3HPE4EHR7MFLYJC5BA, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.642976263 (567h5m13.135408711s) [noise=true , address=45.91.116.4:4938 , piece_id=KZ5Q2APHJV2IPHP6YEN5DDSHNUG7ZZVQBMXVRO3RUNZ7YFJXN4UA, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.643444818 (567h5m13.135877269s) [noise=true , address=145.79.227.14:4837 , piece_id=5WPV4K3J6YFQZH3W3VR4LGZOZLRHGCE6FRMGIKBL7KHYPN2UKT7A, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.644040569 (567h5m13.136473022s) [noise=true , address=77.90.31.10:4518 , piece_id=AJMBNZMOL7SHZYJ5AGQZOYMMNEMJSY4TGU7XWF55QD2GDI2X3GKQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.644599934 (567h5m13.137032387s) [noise=true , address=45.91.116.13:4155 , piece_id=SH5XSK6E6E4J6DSIRDHLTTI3LDBLGC4RQN5DH3KUNTULCXYSK7CA, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.645088026 (567h5m13.137520473s) [noise=true , address=77.90.31.11:4554 , piece_id=6XYEK6IVGFJ34VQGCVJLL2VXFQPR3ZX3YYR6HQ4Y5SL3DZDZFRLQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.645564133 (567h5m13.137996586s) [noise=true , address=145.79.227.15:4874 , piece_id=OW4NLLPLNHKBZ4QAABZH7FS2OKH5L3A536MMZ6ZKGGTMTQJK37NA, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.646066535 (567h5m13.138498988s) [noise=true , address=45.91.116.4:4940 , piece_id=55AISF6FOPT7MNNLIAABGE4DV63WINFTF3W2OTEBOZ5I6IYIS4EQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.646573918 (567h5m13.139006371s) [noise=true , address=145.79.227.3:4151 , piece_id=5HCE6GHWKBSQR6SSKECA5QNU4Y23OVZES6KWFAIHAQVZG4HUHDQQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.647069400 (567h5m13.139501849s) [noise=true , address=77.90.31.17:4771 , piece_id=OXDGHHWYZHXRYAC3P4TPDKMRTQU3DTRDYGRWK7QQZYGZCEG7YZUQ, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:52.647555683 (567h5m13.139988132s) [noise=true , address=77.90.31.13:4628 , piece_id=SE7UJEWLCBGM3N6OCXLSV655MSVBH4R66VMN4IJFOV2LXOMHU6QA, seg_pos={0 0}, upload=498863]: Uploading piece... Sep 22 05:12:53.263869405 (567h5m13.756301841s) [upload=498863]: Next returned nil segment Sep 22 05:12:53.263880756 (567h5m13.7563132s) [upload=498863]: Waiting for error group managing segments... Sep 22 05:12:53.385550938 (567h5m13.877983374s) [noise=true , address=45.91.116.14:4189 , piece_id=JLEHWHBNZTKTYM4OAWCY44TMZYLNEG5A7KGXMCABB5Y677VPK42Q, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.385772818 (567h5m13.878205252s) [noise=true , address=45.91.116.4:4938 , piece_id=KZ5Q2APHJV2IPHP6YEN5DDSHNUG7ZZVQBMXVRO3RUNZ7YFJXN4UA, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.388794490 (567h5m13.881226918s) [noise=true , address=77.90.31.14:4663 , piece_id=KLHWWAPPYZXKDJJ7U3PGGQ4PGBFFP35TLTQOCBRKXIOEDR3LCMZQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.399230502 (567h5m13.891662936s) [noise=true , address=45.91.116.7:4009 , piece_id=HPJJTRLI43FLOMJCVQ4R2YT6F6WQBWTCZV3HPE4EHR7MFLYJC5BA, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.399805338 (567h5m13.892237775s) [noise=true , address=45.91.116.11:4117 , piece_id=63J6DYPINT6TU75XTD4S7EZ7SVNVKKPADERJ3K4463QHCRO7GPPQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.401048161 (567h5m13.89348059s) [noise=true , address=45.91.116.13:4155 , piece_id=SH5XSK6E6E4J6DSIRDHLTTI3LDBLGC4RQN5DH3KUNTULCXYSK7CA, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.403099157 (567h5m13.895531593s) [noise=true , address=45.91.116.4:4940 , piece_id=55AISF6FOPT7MNNLIAABGE4DV63WINFTF3W2OTEBOZ5I6IYIS4EQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.404472930 (567h5m13.896905366s) [noise=true , address=77.90.31.10:4518 , piece_id=AJMBNZMOL7SHZYJ5AGQZOYMMNEMJSY4TGU7XWF55QD2GDI2X3GKQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.411272720 (567h5m13.903705172s) [noise=true , address=77.90.31.11:4554 , piece_id=6XYEK6IVGFJ34VQGCVJLL2VXFQPR3ZX3YYR6HQ4Y5SL3DZDZFRLQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.412105971 (567h5m13.904538406s) [noise=true , address=77.90.31.18:4805 , piece_id=JHBXQ6UJN3GN5YHF6ODWWKCY7DDNNDEL3HCXBAJHAJVPYBJYHYGQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.429011790 (567h5m13.921444233s) [noise=true , address=77.90.31.17:4771 , piece_id=OXDGHHWYZHXRYAC3P4TPDKMRTQU3DTRDYGRWK7QQZYGZCEG7YZUQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:53.429505453 (567h5m13.921937887s) [noise=true , address=77.90.31.13:4628 , piece_id=SE7UJEWLCBGM3N6OCXLSV655MSVBH4R66VMN4IJFOV2LXOMHU6QA, seg_pos={0 0}, upload=498863]: Done uploading piece. err: <nil> Sep 22 05:12:55.059778231 (567h5m15.552210674s) [noise=true , address=145.79.227.16:4909 , piece_id=Z5WGVCFFQ6GUXEIBQXKOZQDZM3E5RRPXXE6BLSBA7LVODAWWVLYA, seg_pos={0 0}, upload=498863]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:13:22.242576479 (567h5m42.735008921s) [noise=true , address=145.79.227.15:4874 , piece_id=OW4NLLPLNHKBZ4QAABZH7FS2OKH5L3A536MMZ6ZKGGTMTQJK37NA, seg_pos={0 0}, upload=498863]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: context deadline exceeded; context deadline exceeded Sep 22 05:13:22.242618475 (567h5m42.73505091s) [noise=true , address=145.79.227.3:4151 , piece_id=5HCE6GHWKBSQR6SSKECA5QNU4Y23OVZES6KWFAIHAQVZG4HUHDQQ, seg_pos={0 0}, upload=498863]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; context deadline exceeded Sep 22 05:13:22.242696858 (567h5m42.735129302s) [noise=true , address=145.79.227.14:4837 , piece_id=5WPV4K3J6YFQZH3W3VR4LGZOZLRHGCE6FRMGIKBL7KHYPN2UKT7A, seg_pos={0 0}, upload=498863]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: context deadline exceeded; context deadline exceeded Sep 22 05:13:22.242746074 (567h5m42.735178527s) [seg_pos={0 0}, upload=498863]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:13:22.350894871 (567h5m42.843327317s) [upload=498863]: Done uploading segments... Sep 22 05:13:22.350927195 (567h5m42.843359644s) [upload=498863]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.582850619s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.54638925s FAIL (2025-09-22 05:13:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ObjectBegin Sep 22 05:05:34.373794494 (566h57m54.866226936s) [upload=498759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:05:34.591982663 (566h57m55.08441511s) [seg_pos={0 0}, upload=498759]: Begin upload segment... Sep 22 05:05:34.592084556 (566h57m55.084517052s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593039674 (566h57m55.08547212s) [seg_pos={0 0}, upload=498759]: Done begin upload segment. Sep 22 05:05:34.593196469 (566h57m55.085628898s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.593794312 (566h57m55.086226761s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594151688 (566h57m55.08658414s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594340355 (566h57m55.086772784s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594638370 (566h57m55.087070816s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.594986459 (566h57m55.087418905s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595499345 (566h57m55.087931787s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595520833 (566h57m55.087953288s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595876308 (566h57m55.088308759s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.595908253 (566h57m55.088340702s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596292954 (566h57m55.088725405s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.596711194 (566h57m55.089143643s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597080427 (566h57m55.089512869s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597430372 (566h57m55.089862805s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.597617754 (566h57m55.090050202s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598038150 (566h57m55.090470595s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:34.598361627 (566h57m55.090794076s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Uploading piece... Sep 22 05:05:35.164291673 (566h57m55.656724108s) [upload=498759]: Next returned nil segment Sep 22 05:05:35.164308803 (566h57m55.656741244s) [upload=498759]: Waiting for error group managing segments... Sep 22 05:05:35.301302795 (566h57m55.793735235s) [noise=true , address=77.90.31.11:4579 , piece_id=ESUUKRMP7W7LOUKXBTGLBH24VTBSJQNRVG45M5CDXEVSN6WSGBKQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.301741421 (566h57m55.794173858s) [noise=true , address=45.91.116.14:4217 , piece_id=2STKYM3XB2GQCHDR5Y3UCH7RZSP2S3D4EGJ7RHXW6ZCLDMJ7TCFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.304878741 (566h57m55.797311176s) [noise=true , address=77.90.31.19:4875 , piece_id=KLXOEJRJ3NC6W4ALCKHCAF2ON4L5XALJ7VKXNA7IIHJARRIX7UFA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.305689145 (566h57m55.79812158s) [noise=true , address=77.90.31.21:4903 , piece_id=T3LHN4FLWKZ34RCZMILDXJWRJRGLJANKGJKCPATMRTX6DUBJWWMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.306847119 (566h57m55.799279561s) [noise=true , address=77.90.31.15:4732 , piece_id=UUY4BGXEWBOLJJ5PNRCZRR6O256XGHMMLSMCNNK7XFOO7QY2X5VA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.339666382 (566h57m55.832098823s) [noise=true , address=145.79.227.46:4608 , piece_id=6ICK2FMUVZW73Y7VNRJ7QM6GSD5VVVCRHOSBT4SZ23TPRZOB5MTQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.347651366 (566h57m55.84008381s) [noise=true , address=145.79.227.46:4621 , piece_id=QCMA65ZBY2VLL3WIDZZZBGSSNJZCAV4CBDL7BUNGRP4TUFJG6X5A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.354716468 (566h57m55.847148904s) [noise=true , address=145.79.227.39:4048 , piece_id=5UGUP5GNOHRPXDREC365MI24LKJDALRSZW6IW3TXQRYAUPPXJOAA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.355447946 (566h57m55.847880382s) [noise=true , address=145.79.227.47:4286 , piece_id=P3XQ3X67SNUIJ73JRPQ36AW36G4CJYCGO6KXU5K3VLXJF6JIZKKA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.364679620 (566h57m55.857112104s) [noise=true , address=145.79.227.40:4082 , piece_id=HKJTQJUSTMKP3YKILUCGM5LMOQKWKTNVNAIUX57LEDN3PHHNBCMQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.371887700 (566h57m55.864320128s) [noise=true , address=145.79.227.49:4340 , piece_id=RZCGUPCF3IYA7XHJV2W67QCTXUXIU7JPB3V5SZ53CEEZAKG5YWDA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:35.847984474 (566h57m56.340416916s) [noise=true , address=66.23.193.175:4555 , piece_id=3WWBSWVBWKORPJ4LEITMLLTOUC437H4J3WDZC3G5TFBAP4SDHTTA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: <nil> Sep 22 05:05:36.462717192 (566h57m56.955149625s) [noise=true , address=145.79.227.11:4757 , piece_id=XQNYJQYY6KPLPLGRB2E2XW2RCUHIKYJKSXMHGS6MCGHS4CKBLT3A, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1uVPbSLBfJdPZqh4yFcEMQTPUwbLhT4GAM813h2nK5qa9xCw9q, address:145.79.227.11:4757): protocol: rpc: dial tcp 145.79.227.11:4757: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068422298 (566h58m54.560854733s) [noise=true , address=145.79.227.14:4867 , piece_id=2YHOBYITZCAT4ZM3SLIMG73VUHJDKPHUTBAGN4GJYEONHJOCN5GQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12Sz7PyMo81CmAAkiCwpZ3M3JaM9HMHJLwAAM5M5kv6nnKktpKE, address:145.79.227.14:4867): protocol: rpc: dial tcp 145.79.227.14:4867: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068467561 (566h58m54.560899994s) [noise=true , address=145.79.227.8:4362 , piece_id=G7YEUATSCT6IA5I7HUKTVPKMN47PTXWGDLFXJWLZAEFKWLU6QWNQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:123zeRoJ5g7juzsV7qQeNacC5BB7jZpfDBnznFhySGWfyPGJ4oa, address:145.79.227.8:4362): protocol: rpc: dial tcp 145.79.227.8:4362: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068538402 (566h58m54.560970833s) [noise=true , address=145.79.227.18:4016 , piece_id=RK2WJCL5M6JXFKO6GIVG6FUB6DRC67UV5HAZUU4V4TB7OSFYTUIQ, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:12vV7puUjzyuxvoUYvc8dASE6bQwPTFeZSW4ZakTbBYWDfGtQoV, address:145.79.227.18:4016): protocol: rpc: dial tcp 145.79.227.18:4016: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068585810 (566h58m54.561018241s) [noise=true , address=145.79.227.13:4815 , piece_id=Z46IWJJTXA6ZJQ2JKRJKCKL3X56I4EXDO6LFO35JWJXJRKMFZTJA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:145.79.227.13:4815): protocol: rpc: dial tcp 145.79.227.13:4815: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068608648 (566h58m54.561041081s) [noise=true , address=145.79.227.10:4433 , piece_id=HNUJP4XBDCZETC2D5AFVUOJKALUNEADYPOJ33JN2JUQKYCQI5VSA, seg_pos={0 0}, upload=498759]: Done uploading piece. err: ecclient: upload failed (node:1VWA9MZwAMNCVwSpoz147FAJRJqP8oVjpqLXtd3oBiyC7t4jsL, address:145.79.227.10:4433): protocol: rpc: dial tcp 145.79.227.10:4433: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:06:34.068628516 (566h58m54.561060949s) [seg_pos={0 0}, upload=498759]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:06:34.184094759 (566h58m54.676527195s) [upload=498759]: Done uploading segments... Sep 22 05:06:34.184123987 (566h58m54.676556429s) [upload=498759]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 2.546422624s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)