Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............................XX.X..................X.X....] duration: 26.936151039s FAIL (2025-07-26 22:09:18 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 26.936272739s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 3.726494732s FAIL (2025-07-26 22:08:59 UTC) orbiter warning: context deadline exceeded Duration: 3.726516179s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 21.667879428s FAIL (2025-07-26 22:08:41 UTC) orbiter warning: context deadline exceeded Duration: 21.667939924s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [......................................X.X..X..XXX....X.X.X..] duration: 3.343860032s FAIL (2025-07-26 22:08:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 3.343899603s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 51.830740198s FAIL (2025-07-26 22:08:31 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 51.830774967s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................XX.X..................X.X...] duration: 2.733473102s FAIL (2025-07-26 22:07:53 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.733496977s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 3.21160947s FAIL (2025-07-26 22:07:53 UTC) orbiter warning: context deadline exceeded Duration: 3.211634738s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................XX.X..................X.X..] duration: 27.886960951s FAIL (2025-07-26 22:07:50 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 27.887007505s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.......................................X.X..X..XXX....X.X.X.] duration: 29.042064462s FAIL (2025-07-26 22:07:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 29.042085934s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 4.11993944s FAIL (2025-07-26 22:07:35 UTC) orbiter warning: context deadline exceeded Duration: 4.119963643s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 38.201512783s FAIL (2025-07-26 22:07:28 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 38.2015453s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [........................................X.X..X..XXX....X.X.X] duration: 30.114032384s FAIL (2025-07-26 22:07:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== [upload=56111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:50.934818576 (58h30m22.125117166s) [upload=56111]: Done uploading segments... Jul 26 22:05:50.934843990 (58h30m22.125142556s) [upload=56111]: Upload finished. err: <nil> Jul 26 22:06:43.410531028 (58h31m14.60082962s) [upload=56129]: Starting upload Jul 26 22:06:43.410634062 (58h31m14.600932642s) [upload=56129]: Done starting upload Jul 26 22:06:43.410776507 (58h31m14.601075088s) [upload=56129]: Uploading segments... Jul 26 22:06:43.411226751 (58h31m14.60152534s) [upload=56129]: Got next segment. Inline: false Jul 26 22:06:43.411314789 (58h31m14.60161338s) [upload=56129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:06:43.411343160 (58h31m14.60164175s) [upload=56129]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:06:43.522292825 (58h31m14.712591404s) [seg_pos={0 0}, upload=56129]: Begin upload segment... Jul 26 22:06:43.522343754 (58h31m14.712642334s) [seg_pos={0 0}, upload=56129]: Done begin upload segment. Jul 26 22:06:43.522375894 (58h31m14.712674484s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522391089 (58h31m14.712689677s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.522860551 (58h31m14.713159135s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523006961 (58h31m14.713305548s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523207242 (58h31m14.713505828s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523503643 (58h31m14.713802228s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523520990 (58h31m14.713819575s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523801442 (58h31m14.714100028s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.523875913 (58h31m14.714174502s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524090580 (58h31m14.714389165s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524561989 (58h31m14.714860573s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524621764 (58h31m14.714920352s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.524933165 (58h31m14.715231753s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525071352 (58h31m14.715369943s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.525266749 (58h31m14.715565334s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:43.526743137 (58h31m14.717041709s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Uploading piece... Jul 26 22:06:44.133279254 (58h31m15.323577833s) [upload=56129]: Next returned nil segment Jul 26 22:06:44.133307297 (58h31m15.323605886s) [upload=56129]: Waiting for error group managing segments... Jul 26 22:06:44.272332326 (58h31m15.462630911s) [noise=true , address=45.91.116.8:4047 , piece_id=ZTUCNUUUU3HOTPUQIG56LJDAKHVTF6XRY7IALIQESXADJVPS3Y6A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272387683 (58h31m15.462686284s) [noise=true , address=45.91.116.7:4009 , piece_id=GXGL4S7X63QFJ6RQ2X5ZOD7LWUTD57VD2WILBMQR7TAURAZOIOMQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.272988407 (58h31m15.463286973s) [noise=true , address=45.91.116.4:4939 , piece_id=NA2OWU4QW2LYCYRGISJLKSQZ37D6KCXC2UDUBHG3Z3B2XZ6L5XAQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.273174988 (58h31m15.463473559s) [noise=true , address=45.91.116.12:4225 , piece_id=M3X35ZQFBN3A6KE3JNFLUXSESEEA7L73SLJMX523JEK5R4I2EB4A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275175424 (58h31m15.465473998s) [noise=true , address=77.90.31.19:4843 , piece_id=IRHAWUHUCRJ4INLRQRDATY5BYRAKYOCWNUZM5CBJTD7RR2YZEJZA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275534117 (58h31m15.465832702s) [noise=true , address=45.91.116.11:4117 , piece_id=OWYFLQEVHN3S3N7NVGU2T6N3JDLXXAWZGE7WS2UKMDJ6SQLQ2QNA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.275720118 (58h31m15.466018687s) [noise=true , address=45.91.116.8:4048 , piece_id=EEWAE5AYN7EWGTT64NXA4R4HUBB37BNBLIDMGL3DELBT7GA6B5HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.276477888 (58h31m15.466776468s) [noise=true , address=45.91.116.10:4081 , piece_id=I6NOVU7GVKGFAP7IRLUSM2XBGY6TGJJXLBOF5J7XSTSQBA7D52HA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.281633280 (58h31m15.471931848s) [noise=true , address=45.91.116.9:4262 , piece_id=N5Y4NN7EBQVDIN7KCKINKSUNBFQ4GGJE5YOK7B3YNE4WJYHX65VQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.282497510 (58h31m15.472796096s) [noise=true , address=77.90.31.21:4877 , piece_id=MNWB4UFSKZ4UFXZ5HHXNTSAUCWX4ZY2AYQUU5RSZMFMICISQXHWA, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:06:44.290894855 (58h31m15.481193505s) [noise=true , address=45.91.116.14:4191 , piece_id=7GVSIJ7WGFK3KOOAOYXJ5KHMI3AF5LQLRYAOYEE6MJ4HFIEOZ4SQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: <nil> Jul 26 22:07:13.142915449 (58h31m44.333214032s) [noise=true , address=104.36.82.4:4803 , piece_id=YR5PXA7GC473FTVLMU35RWNRVPVRAIQKHBMLLRNLW2FAPYSNMMKQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 22:07:13.143197886 (58h31m44.333496484s) [noise=true , address=104.36.82.8:4972 , piece_id=LFINCN7K4UGLR6E45EOQITLY552JAIVOFK2M2G5DTALYME4BQFNQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50904->104.36.82.8:4972: use of closed network connection Jul 26 22:07:13.143265025 (58h31m44.333563606s) [noise=true , address=104.36.82.3:4767 , piece_id=OHPFLAPHO6YNS7AAWYCPWJ7TOS4QX5WUKBZC56LBBY6DL3SJMY5A, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50890->104.36.82.3:4767: use of closed network connection Jul 26 22:07:13.143280600 (58h31m44.333579187s) [noise=true , address=77.90.31.10:4518 , piece_id=JQTOSUUKKAQH7YR6ZPLHE6NN74UDDYF64KXYY7MUFNTVFWSSPD3Q, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 139.59.144.22:58568->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 22:07:13.143340504 (58h31m44.333639094s) [noise=true , address=104.36.82.8:4970 , piece_id=6ME3LTB6DA4E2SJR6WVUYCZAWZYY7FPT3Q67NJZ5NXWMF2YRY4DQ, seg_pos={0 0}, upload=56129]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58088->104.36.82.8:4970: use of closed network connection Jul 26 22:07:13.143399908 (58h31m44.333698489s) [seg_pos={0 0}, upload=56129]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 22:07:13.255165693 (58h31m44.445464263s) [upload=56129]: Done uploading segments... Jul 26 22:07:13.255194664 (58h31m44.445493243s) [upload=56129]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration:* 30.114058116s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.915637968s FAIL (2025-07-26 22:06:36 UTC) orbiter warning: context deadline exceeded Duration: 2.915667261s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 44.489747981s FAIL (2025-07-26 22:06:21 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 44.489778413s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.771886681s FAIL (2025-07-26 22:06:13 UTC) orbiter warning: context deadline exceeded Duration: 2.771915533s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................XX.X..................X.X.] duration: 2.287835652s FAIL (2025-07-26 22:06:12 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.287874884s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................................XX.X..................X.X] duration: 30.143589493s FAIL (2025-07-26 22:06:09 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.143615627s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........................................X.X..X..XXX....X.X.] duration: 19.602382947s FAIL (2025-07-26 22:05:51 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== nceled Jul 26 22:04:09.421631207 (58h28m40.611929803s) [seg_pos={0 0}, upload=56085]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 22:04:09.421734892 (58h28m40.612033459s) [upload=56085]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:04:09.421739273 (58h28m40.61203784s) [upload=56085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:04:09.569890041 (58h28m40.760188634s) [upload=56085]: Done uploading segments... Jul 26 22:04:09.569955179 (58h28m40.760253756s) [upload=56085]: Upload finished. err: <nil> Jul 26 22:04:56.207147247 (58h29m27.397445842s) [upload=56102]: Starting upload Jul 26 22:04:56.207274986 (58h29m27.397573585s) [upload=56102]: Done starting upload Jul 26 22:04:56.207488330 (58h29m27.397786918s) [upload=56102]: Uploading segments... Jul 26 22:04:56.207960175 (58h29m27.398258757s) [upload=56102]: Got next segment. Inline: false Jul 26 22:04:56.208003005 (58h29m27.398301585s) [upload=56102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:04:56.208011042 (58h29m27.398309623s) [upload=56102]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:04:56.313865267 (58h29m27.504163837s) [seg_pos={0 0}, upload=56102]: Begin upload segment... Jul 26 22:04:56.313957005 (58h29m27.504255574s) [seg_pos={0 0}, upload=56102]: Done begin upload segment. Jul 26 22:04:56.313986475 (58h29m27.504285059s) [noise=true , address=104.36.82.9:4006 , piece_id=5GKAO4QT73WEYGKALRLHKQ6W7TUBDRWWAPXBPVWJDYYQIL2YLCFQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314034753 (58h29m27.50433334s) [noise=true , address=45.91.116.10:4081 , piece_id=EX75BQ4FWBINYPMI2SD3HM2DJ6ASNWKDTGIVDP653HBNEM2GJAVA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314460472 (58h29m27.504759057s) [noise=true , address=77.90.31.13:4628 , piece_id=NQCGYNAMSX3ORJXPLBRWK45ZKUF2KKV2IGGNQ7DYRS44U43EXXEQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314488328 (58h29m27.504786915s) [noise=true , address=77.90.31.13:4627 , piece_id=OP4CQ4HZUADBL3IHEUKXWBPOGC7NWNVIWFLKJF6ZC6LHEPUSZOLQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314835120 (58h29m27.505133741s) [noise=true , address=45.91.116.5:4298 , piece_id=DFZZPA4XLYCFYJNKKO3UCU47YQML54WAZ7U5IHYZFEXCG7L5IVOQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314849965 (58h29m27.505148548s) [noise=true , address=77.90.31.10:4519 , piece_id=CSYDU7PWXB5FWGQBGYWXBUCLLJYNWLLRJ22COLVPXPGYD27DIAIA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315217159 (58h29m27.505515748s) [noise=true , address=45.91.116.11:4119 , piece_id=ZO4I3GKQIUWJOEXH6UB6DRAE4UWMGFNE3SFMMBWMTON6A6XAZZLA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315224792 (58h29m27.505523372s) [noise=true , address=45.91.116.8:4048 , piece_id=S7TRCUKU6XJJIMHPUJK4K27KLIGUOQNCX7LSJOQQFSPMJDA5QBPQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315592819 (58h29m27.505891403s) [noise=true , address=45.91.116.12:4226 , piece_id=XLFA7M74ODTWQ6AKK26WEHBULXJRTQQSJ4HMTTOZRPCFJBOQ4AIQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315821540 (58h29m27.506120131s) [noise=true , address=45.91.116.4:4938 , piece_id=MPS52JMUGTZX5LCLCCGC6O34OSDLZRTDXX5ACECYONXBZI7S5XLA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315950277 (58h29m27.506248848s) [noise=true , address=45.91.116.13:4154 , piece_id=YVK4JJAHXMJKVQPQQRBDK3KSLT2XNPCIKJ2BYAKPWUYZR23SA6TA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316455053 (58h29m27.506753637s) [noise=true , address=45.91.116.7:4009 , piece_id=5QNL7F4ALJLAZOEGTISN74M74QA2VFGE5JQBHKGU3Q4OTWEQHWUA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316510075 (58h29m27.506808661s) [noise=true , address=104.36.82.8:4971 , piece_id=X3FPZ2JVEUVHQSX6FM7UM5APOLMAD5S2APWHPIGS35KZXI77WP4Q, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316804724 (58h29m27.507103308s) [noise=true , address=45.91.116.6:4973 , piece_id=YR5SXI2XFILSLJ5BNIV3ZPAUJ22I2QBXJ6IGGCTDLHAWUQTZCJZQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316891708 (58h29m27.507190289s) [noise=true , address=104.36.82.4:4802 , piece_id=V4GOSED4TSJLLNV6MJMQND5CSJN6X5SQSETCOC4SUIU6V2OASBXQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.317272029 (58h29m27.507570633s) [noise=true , address=104.36.82.14:4221 , piece_id=2D52DCCQMIAYPHPRBYVZ3SV26Q7TKWSFAMLWJ7Y22EI62N37YTHA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.926481147 (58h29m28.116779736s) [upload=56102]: Next returned nil segment Jul 26 22:04:56.926506432 (58h29m28.116805022s) [upload=56102]: Waiting for error group managing segments... Jul 26 22:04:57.066673832 (58h29m28.256972416s) [noise=true , address=45.91.116.10:4081 , piece_id=EX75BQ4FWBINYPMI2SD3HM2DJ6ASNWKDTGIVDP653HBNEM2GJAVA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.066825188 (58h29m28.257123775s) [noise=true , address=45.91.116.6:4973 , piece_id=YR5SXI2XFILSLJ5BNIV3ZPAUJ22I2QBXJ6IGGCTDLHAWUQTZCJZQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.068592765 (58h29m28.258891335s) [noise=true , address=45.91.116.13:4154 , piece_id=YVK4JJAHXMJKVQPQQRBDK3KSLT2XNPCIKJ2BYAKPWUYZR23SA6TA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.068677659 (58h29m28.258976233s) [noise=true , address=45.91.116.4:4938 , piece_id=MPS52JMUGTZX5LCLCCGC6O34OSDLZRTDXX5ACECYONXBZI7S5XLA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.069601195 (58h29m28.259899776s) [noise=true , address=45.91.116.7:4009 , piece_id=5QNL7F4ALJLAZOEGTISN74M74QA2VFGE5JQBHKGU3Q4OTWEQHWUA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.069995788 (58h29m28.260294356s) [noise=true , address=45.91.116.8:4048 , piece_id=S7TRCUKU6XJJIMHPUJK4K27KLIGUOQNCX7LSJOQQFSPMJDA5QBPQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.073537823 (58h29m28.263836403s) [noise=true , address=45.91.116.12:4226 , piece_id=XLFA7M74ODTWQ6AKK26WEHBULXJRTQQSJ4HMTTOZRPCFJBOQ4AIQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.074381893 (58h29m28.264680461s) [noise=true , address=45.91.116.11:4119 , piece_id=ZO4I3GKQIUWJOEXH6UB6DRAE4UWMGFNE3SFMMBWMTON6A6XAZZLA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.082741327 (58h29m28.273039901s) [noise=true , address=45.91.116.5:4298 , piece_id=DFZZPA4XLYCFYJNKKO3UCU47YQML54WAZ7U5IHYZFEXCG7L5IVOQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:02.570015392 (58h29m33.760313986s) [noise=true , address=77.90.31.10:4519 , piece_id=CSYDU7PWXB5FWGQBGYWXBUCLLJYNWLLRJ22COLVPXPGYD27DIAIA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:06.934646421 (58h29m38.124945001s) [noise=true , address=77.90.31.13:4628 , piece_id=NQCGYNAMSX3ORJXPLBRWK45ZKUF2KKV2IGGNQ7DYRS44U43EXXEQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:07.040246299 (58h29m38.23054487s) [noise=true , address=104.36.82.4:4802 , piece_id=V4GOSED4TSJLLNV6MJMQND5CSJN6X5SQSETCOC4SUIU6V2OASBXQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:14.247749226 (58h29m45.438047791s) [noise=true , address=77.90.31.13:4627 , piece_id=OP4CQ4HZUADBL3IHEUKXWBPOGC7NWNVIWFLKJF6ZC6LHEPUSZOLQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:14.247815734 (58h29m45.438114318s) [seg_pos={0 0}, upload=56102]: Segment reached optimal threshold of 13 pieces. Jul 26 22:05:14.248091094 (58h29m45.438389664s) [noise=true , address=104.36.82.9:4006 , piece_id=5GKAO4QT73WEYGKALRLHKQ6W7TUBDRWWAPXBPVWJDYYQIL2YLCFQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF Jul 26 22:05:14.248277239 (58h29m45.438575815s) [noise=true , address=104.36.82.14:4221 , piece_id=2D52DCCQMIAYPHPRBYVZ3SV26Q7TKWSFAMLWJ7Y22EI62N37YTHA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 139.59.144.22:40552->104.36.82.14:4221: use of closed network connection; write tcp 139.59.144.22:40552->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40552->104.36.82.14:4221: use of closed network connection Jul 26 22:05:14.248443049 (58h29m45.438741618s) [noise=true , address=104.36.82.8:4971 , piece_id=X3FPZ2JVEUVHQSX6FM7UM5APOLMAD5S2APWHPIGS35KZXI77WP4Q, seg_pos={0 0}, upload=56102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 139.59.144.22:54752->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:54752->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54752->104.36.82.8:4971: use of closed network connection Jul 26 22:05:14.248477076 (58h29m45.438775645s) [seg_pos={0 0}, upload=56102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 22:05:14.248574491 (58h29m45.438873072s) [upload=56102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:05:14.248581528 (58h29m45.438880107s) [upload=56102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:14.509360242 (58h29m45.699658828s) [upload=56102]: Done uploading segments... Jul 26 22:05:14.509383528 (58h29m45.699682108s) [upload=56102]: Upload finished. err: <nil> Duration: 19.602417685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 51.714228382s FAIL (2025-07-26 22:05:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 51.714276604s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........................................X.X..X..XXX....X.X] duration: 30.002526783s FAIL (2025-07-26 22:05:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== nceled Jul 26 22:04:09.421631207 (58h28m40.611929803s) [seg_pos={0 0}, upload=56085]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 22:04:09.421734892 (58h28m40.612033459s) [upload=56085]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:04:09.421739273 (58h28m40.61203784s) [upload=56085]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:04:09.569890041 (58h28m40.760188634s) [upload=56085]: Done uploading segments... Jul 26 22:04:09.569955179 (58h28m40.760253756s) [upload=56085]: Upload finished. err: <nil> Jul 26 22:04:56.207147247 (58h29m27.397445842s) [upload=56102]: Starting upload Jul 26 22:04:56.207274986 (58h29m27.397573585s) [upload=56102]: Done starting upload Jul 26 22:04:56.207488330 (58h29m27.397786918s) [upload=56102]: Uploading segments... Jul 26 22:04:56.207960175 (58h29m27.398258757s) [upload=56102]: Got next segment. Inline: false Jul 26 22:04:56.208003005 (58h29m27.398301585s) [upload=56102]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:04:56.208011042 (58h29m27.398309623s) [upload=56102]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:04:56.313865267 (58h29m27.504163837s) [seg_pos={0 0}, upload=56102]: Begin upload segment... Jul 26 22:04:56.313957005 (58h29m27.504255574s) [seg_pos={0 0}, upload=56102]: Done begin upload segment. Jul 26 22:04:56.313986475 (58h29m27.504285059s) [noise=true , address=104.36.82.9:4006 , piece_id=5GKAO4QT73WEYGKALRLHKQ6W7TUBDRWWAPXBPVWJDYYQIL2YLCFQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314034753 (58h29m27.50433334s) [noise=true , address=45.91.116.10:4081 , piece_id=EX75BQ4FWBINYPMI2SD3HM2DJ6ASNWKDTGIVDP653HBNEM2GJAVA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314460472 (58h29m27.504759057s) [noise=true , address=77.90.31.13:4628 , piece_id=NQCGYNAMSX3ORJXPLBRWK45ZKUF2KKV2IGGNQ7DYRS44U43EXXEQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314488328 (58h29m27.504786915s) [noise=true , address=77.90.31.13:4627 , piece_id=OP4CQ4HZUADBL3IHEUKXWBPOGC7NWNVIWFLKJF6ZC6LHEPUSZOLQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314835120 (58h29m27.505133741s) [noise=true , address=45.91.116.5:4298 , piece_id=DFZZPA4XLYCFYJNKKO3UCU47YQML54WAZ7U5IHYZFEXCG7L5IVOQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.314849965 (58h29m27.505148548s) [noise=true , address=77.90.31.10:4519 , piece_id=CSYDU7PWXB5FWGQBGYWXBUCLLJYNWLLRJ22COLVPXPGYD27DIAIA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315217159 (58h29m27.505515748s) [noise=true , address=45.91.116.11:4119 , piece_id=ZO4I3GKQIUWJOEXH6UB6DRAE4UWMGFNE3SFMMBWMTON6A6XAZZLA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315224792 (58h29m27.505523372s) [noise=true , address=45.91.116.8:4048 , piece_id=S7TRCUKU6XJJIMHPUJK4K27KLIGUOQNCX7LSJOQQFSPMJDA5QBPQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315592819 (58h29m27.505891403s) [noise=true , address=45.91.116.12:4226 , piece_id=XLFA7M74ODTWQ6AKK26WEHBULXJRTQQSJ4HMTTOZRPCFJBOQ4AIQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315821540 (58h29m27.506120131s) [noise=true , address=45.91.116.4:4938 , piece_id=MPS52JMUGTZX5LCLCCGC6O34OSDLZRTDXX5ACECYONXBZI7S5XLA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.315950277 (58h29m27.506248848s) [noise=true , address=45.91.116.13:4154 , piece_id=YVK4JJAHXMJKVQPQQRBDK3KSLT2XNPCIKJ2BYAKPWUYZR23SA6TA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316455053 (58h29m27.506753637s) [noise=true , address=45.91.116.7:4009 , piece_id=5QNL7F4ALJLAZOEGTISN74M74QA2VFGE5JQBHKGU3Q4OTWEQHWUA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316510075 (58h29m27.506808661s) [noise=true , address=104.36.82.8:4971 , piece_id=X3FPZ2JVEUVHQSX6FM7UM5APOLMAD5S2APWHPIGS35KZXI77WP4Q, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316804724 (58h29m27.507103308s) [noise=true , address=45.91.116.6:4973 , piece_id=YR5SXI2XFILSLJ5BNIV3ZPAUJ22I2QBXJ6IGGCTDLHAWUQTZCJZQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.316891708 (58h29m27.507190289s) [noise=true , address=104.36.82.4:4802 , piece_id=V4GOSED4TSJLLNV6MJMQND5CSJN6X5SQSETCOC4SUIU6V2OASBXQ, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.317272029 (58h29m27.507570633s) [noise=true , address=104.36.82.14:4221 , piece_id=2D52DCCQMIAYPHPRBYVZ3SV26Q7TKWSFAMLWJ7Y22EI62N37YTHA, seg_pos={0 0}, upload=56102]: Uploading piece... Jul 26 22:04:56.926481147 (58h29m28.116779736s) [upload=56102]: Next returned nil segment Jul 26 22:04:56.926506432 (58h29m28.116805022s) [upload=56102]: Waiting for error group managing segments... Jul 26 22:04:57.066673832 (58h29m28.256972416s) [noise=true , address=45.91.116.10:4081 , piece_id=EX75BQ4FWBINYPMI2SD3HM2DJ6ASNWKDTGIVDP653HBNEM2GJAVA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.066825188 (58h29m28.257123775s) [noise=true , address=45.91.116.6:4973 , piece_id=YR5SXI2XFILSLJ5BNIV3ZPAUJ22I2QBXJ6IGGCTDLHAWUQTZCJZQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.068592765 (58h29m28.258891335s) [noise=true , address=45.91.116.13:4154 , piece_id=YVK4JJAHXMJKVQPQQRBDK3KSLT2XNPCIKJ2BYAKPWUYZR23SA6TA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.068677659 (58h29m28.258976233s) [noise=true , address=45.91.116.4:4938 , piece_id=MPS52JMUGTZX5LCLCCGC6O34OSDLZRTDXX5ACECYONXBZI7S5XLA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.069601195 (58h29m28.259899776s) [noise=true , address=45.91.116.7:4009 , piece_id=5QNL7F4ALJLAZOEGTISN74M74QA2VFGE5JQBHKGU3Q4OTWEQHWUA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.069995788 (58h29m28.260294356s) [noise=true , address=45.91.116.8:4048 , piece_id=S7TRCUKU6XJJIMHPUJK4K27KLIGUOQNCX7LSJOQQFSPMJDA5QBPQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.073537823 (58h29m28.263836403s) [noise=true , address=45.91.116.12:4226 , piece_id=XLFA7M74ODTWQ6AKK26WEHBULXJRTQQSJ4HMTTOZRPCFJBOQ4AIQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.074381893 (58h29m28.264680461s) [noise=true , address=45.91.116.11:4119 , piece_id=ZO4I3GKQIUWJOEXH6UB6DRAE4UWMGFNE3SFMMBWMTON6A6XAZZLA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:04:57.082741327 (58h29m28.273039901s) [noise=true , address=45.91.116.5:4298 , piece_id=DFZZPA4XLYCFYJNKKO3UCU47YQML54WAZ7U5IHYZFEXCG7L5IVOQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:02.570015392 (58h29m33.760313986s) [noise=true , address=77.90.31.10:4519 , piece_id=CSYDU7PWXB5FWGQBGYWXBUCLLJYNWLLRJ22COLVPXPGYD27DIAIA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:06.934646421 (58h29m38.124945001s) [noise=true , address=77.90.31.13:4628 , piece_id=NQCGYNAMSX3ORJXPLBRWK45ZKUF2KKV2IGGNQ7DYRS44U43EXXEQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:07.040246299 (58h29m38.23054487s) [noise=true , address=104.36.82.4:4802 , piece_id=V4GOSED4TSJLLNV6MJMQND5CSJN6X5SQSETCOC4SUIU6V2OASBXQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:14.247749226 (58h29m45.438047791s) [noise=true , address=77.90.31.13:4627 , piece_id=OP4CQ4HZUADBL3IHEUKXWBPOGC7NWNVIWFLKJF6ZC6LHEPUSZOLQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: <nil> Jul 26 22:05:14.247815734 (58h29m45.438114318s) [seg_pos={0 0}, upload=56102]: Segment reached optimal threshold of 13 pieces. Jul 26 22:05:14.248091094 (58h29m45.438389664s) [noise=true , address=104.36.82.9:4006 , piece_id=5GKAO4QT73WEYGKALRLHKQ6W7TUBDRWWAPXBPVWJDYYQIL2YLCFQ, seg_pos={0 0}, upload=56102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF Jul 26 22:05:14.248277239 (58h29m45.438575815s) [noise=true , address=104.36.82.14:4221 , piece_id=2D52DCCQMIAYPHPRBYVZ3SV26Q7TKWSFAMLWJ7Y22EI62N37YTHA, seg_pos={0 0}, upload=56102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 139.59.144.22:40552->104.36.82.14:4221: use of closed network connection; write tcp 139.59.144.22:40552->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40552->104.36.82.14:4221: use of closed network connection Jul 26 22:05:14.248443049 (58h29m45.438741618s) [noise=true , address=104.36.82.8:4971 , piece_id=X3FPZ2JVEUVHQSX6FM7UM5APOLMAD5S2APWHPIGS35KZXI77WP4Q, seg_pos={0 0}, upload=56102]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 139.59.144.22:54752->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:54752->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54752->104.36.82.8:4971: use of closed network connection Jul 26 22:05:14.248477076 (58h29m45.438775645s) [seg_pos={0 0}, upload=56102]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 22:05:14.248574491 (58h29m45.438873072s) [upload=56102]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:05:14.248581528 (58h29m45.438880107s) [upload=56102]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:05:14.509360242 (58h29m45.699658828s) [upload=56102]: Done uploading segments... Jul 26 22:05:14.509383528 (58h29m45.699682108s) [upload=56102]: Upload finished. err: <nil> Duration: 30.002557498s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 10.475518693s FAIL (2025-07-26 22:05:15 UTC) orbiter warning: context deadline exceeded Duration: 10.475609224s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................XX.X..................X.] duration: 19.065958598s FAIL (2025-07-26 22:04:59 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 19.06598352s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 53.752854499s FAIL (2025-07-26 22:04:45 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 53.752980721s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XX.X..................X] duration: 30.123732587s FAIL (2025-07-26 22:04:40 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.123779445s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...........................................X.X..X..XXX....X.] duration: 15.24616071s FAIL (2025-07-26 22:04:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== (58h27m19.160852834s) [upload=56059]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.970560064 (58h27m19.160858629s) [upload=56059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.117843903 (58h27m19.30814249s) [upload=56059]: Done uploading segments... Jul 26 22:02:48.117867719 (58h27m19.308166287s) [upload=56059]: Upload finished. err: <nil> Jul 26 22:03:16.088194620 (58h27m47.278493235s) [upload=56074]: Starting upload Jul 26 22:03:16.088376905 (58h27m47.27867547s) [upload=56074]: Done starting upload Jul 26 22:03:16.088810540 (58h27m47.279109118s) [upload=56074]: Uploading segments... Jul 26 22:03:16.088950032 (58h27m47.279248621s) [upload=56074]: Got next segment. Inline: false Jul 26 22:03:16.088997348 (58h27m47.279295935s) [upload=56074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:03:16.089003698 (58h27m47.279302287s) [upload=56074]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:03:16.235927985 (58h27m47.426226563s) [seg_pos={0 0}, upload=56074]: Begin upload segment... Jul 26 22:03:16.236008106 (58h27m47.426306692s) [seg_pos={0 0}, upload=56074]: Done begin upload segment. Jul 26 22:03:16.236047175 (58h27m47.426345761s) [noise=true , address=45.91.116.9:4263 , piece_id=GMZLI6VL3E646LGYVEWJ4H6EHKVD6EPETQGNBHCX6GFW4EREUMAQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.236554619 (58h27m47.426853198s) [noise=true , address=45.91.116.6:4973 , piece_id=7RQGHXVEUQIEUZM7DNSCID4CH4CDQZWIRZDYEUFET7LBATT7BPTA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.237546455 (58h27m47.427845037s) [noise=true , address=45.91.116.14:4189 , piece_id=ZCN7AL4H7P7MFIFBK6674FOHFDVOR7F2W4CIUHDL2RUJGDEJNISQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.237726616 (58h27m47.428025194s) [noise=true , address=45.91.116.12:4225 , piece_id=CTHWSYJ4NXGTNGA7DEXX2WHCGDXHAPXCMHVBWUJ6PYGN35MN4I5A, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.239644947 (58h27m47.429943535s) [noise=true , address=45.91.116.10:4082 , piece_id=EZYC47CADGVT2QTYCFJRJGQBVBDK5MK2SONEHCOHYOV43DLJS4GA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.240347271 (58h27m47.430645857s) [noise=true , address=45.91.116.13:4154 , piece_id=S6TYBP5H475KQQAE7RXJLG27BOXIFIRDN3DXLD6GZVFOVYZH46QA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.241494652 (58h27m47.43179324s) [noise=true , address=77.90.31.16:4734 , piece_id=BEOIIBUQVSKBMMIWRUDH3FNMTGDX5H4S2TZUPXEPMEAVF5HWAO3Q, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.242192055 (58h27m47.432490637s) [noise=true , address=104.36.82.3:4765 , piece_id=7F3B5K6MJZB7MZODDN3VSUUOWOREFLKNRHVZC26RWFCSF7P2MXJQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.244691319 (58h27m47.434989909s) [noise=true , address=104.36.82.4:4801 , piece_id=2COQU4P4L7DY2LUPVCTH2DLPCBYUQRKT6PYQ3BB2AVKAZWSWMQOA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.246470562 (58h27m47.436769151s) [noise=true , address=104.36.82.14:4223 , piece_id=GQMFBCVPMPZ7EMHJGNZKBWDZOYMVVIEYFXHM3QSH3O2OCKO7G2BA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.247902260 (58h27m47.438200851s) [noise=true , address=77.90.31.19:4841 , piece_id=BOF2X6NFK5FC5HVAYM7VZV7TELJEOZRWD4GRQPXNGGS32VDTNCIQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.254607394 (58h27m47.444905968s) [noise=true , address=104.36.82.6:4876 , piece_id=EMSUE4AA33FPR2G34KGMEVOOHUIRFAY2RQZQFYY3CZEDOS45AGEQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.268402565 (58h27m47.458701139s) [noise=true , address=45.91.116.11:4118 , piece_id=4RSJU3B3XNBJR3L6E673J4EFGIXRE2Y3PGB24ZKMIU3H5E3JIPUA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.268822159 (58h27m47.459120733s) [noise=true , address=104.36.82.12:4150 , piece_id=5ZIOMTZQBC66A2SRYPO2MHKK6I4S2DCOFQTUSO5K3A5XCYRNSF3Q, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.270290736 (58h27m47.460589309s) [noise=true , address=45.91.116.7:4011 , piece_id=325G6VS4EHSDX7WXXNV3AMJBDM2FTEUATRCGCL7U7WOO5T66S6XQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.270873178 (58h27m47.461171749s) [noise=true , address=45.91.116.5:4299 , piece_id=4Y5AHJVCCP4VWJ4N5PY2WE4DSFLBJVZN3JPYC3V6JVTF6QCPNNZQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.872897695 (58h27m48.063196287s) [upload=56074]: Next returned nil segment Jul 26 22:03:16.872968746 (58h27m48.063267334s) [upload=56074]: Waiting for error group managing segments... Jul 26 22:03:17.007002942 (58h27m48.197301515s) [noise=true , address=45.91.116.14:4189 , piece_id=ZCN7AL4H7P7MFIFBK6674FOHFDVOR7F2W4CIUHDL2RUJGDEJNISQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.007209108 (58h27m48.1975077s) [noise=true , address=45.91.116.10:4082 , piece_id=EZYC47CADGVT2QTYCFJRJGQBVBDK5MK2SONEHCOHYOV43DLJS4GA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.009322862 (58h27m48.19962144s) [noise=true , address=45.91.116.13:4154 , piece_id=S6TYBP5H475KQQAE7RXJLG27BOXIFIRDN3DXLD6GZVFOVYZH46QA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.009602813 (58h27m48.199901394s) [noise=true , address=45.91.116.6:4973 , piece_id=7RQGHXVEUQIEUZM7DNSCID4CH4CDQZWIRZDYEUFET7LBATT7BPTA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.015688237 (58h27m48.205986822s) [noise=true , address=45.91.116.12:4225 , piece_id=CTHWSYJ4NXGTNGA7DEXX2WHCGDXHAPXCMHVBWUJ6PYGN35MN4I5A, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.017443592 (58h27m48.207742183s) [noise=true , address=45.91.116.7:4011 , piece_id=325G6VS4EHSDX7WXXNV3AMJBDM2FTEUATRCGCL7U7WOO5T66S6XQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.019010997 (58h27m48.209309566s) [noise=true , address=45.91.116.5:4299 , piece_id=4Y5AHJVCCP4VWJ4N5PY2WE4DSFLBJVZN3JPYC3V6JVTF6QCPNNZQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.035401373 (58h27m48.225699938s) [noise=true , address=45.91.116.11:4118 , piece_id=4RSJU3B3XNBJR3L6E673J4EFGIXRE2Y3PGB24ZKMIU3H5E3JIPUA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.059860273 (58h27m48.250158857s) [noise=true , address=45.91.116.9:4263 , piece_id=GMZLI6VL3E646LGYVEWJ4H6EHKVD6EPETQGNBHCX6GFW4EREUMAQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:31.769542300 (58h28m2.959840889s) [noise=true , address=77.90.31.19:4841 , piece_id=BOF2X6NFK5FC5HVAYM7VZV7TELJEOZRWD4GRQPXNGGS32VDTNCIQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:42.488922757 (58h28m13.679221324s) [noise=true , address=104.36.82.3:4765 , piece_id=7F3B5K6MJZB7MZODDN3VSUUOWOREFLKNRHVZC26RWFCSF7P2MXJQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:44.654593132 (58h28m15.844891709s) [noise=true , address=77.90.31.16:4734 , piece_id=BEOIIBUQVSKBMMIWRUDH3FNMTGDX5H4S2TZUPXEPMEAVF5HWAO3Q, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:45.817542617 (58h28m17.007841197s) [noise=true , address=104.36.82.12:4150 , piece_id=5ZIOMTZQBC66A2SRYPO2MHKK6I4S2DCOFQTUSO5K3A5XCYRNSF3Q, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:50086->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:50086->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50086->104.36.82.12:4150: use of closed network connection Jul 26 22:03:45.817841045 (58h28m17.008139634s) [noise=true , address=104.36.82.4:4801 , piece_id=2COQU4P4L7DY2LUPVCTH2DLPCBYUQRKT6PYQ3BB2AVKAZWSWMQOA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:42104->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:42104->104.36.82.4:4801: use of closed network connection Jul 26 22:03:45.817962004 (58h28m17.008260576s) [noise=true , address=104.36.82.14:4223 , piece_id=GQMFBCVPMPZ7EMHJGNZKBWDZOYMVVIEYFXHM3QSH3O2OCKO7G2BA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:32942->104.36.82.14:4223: use of closed network connection; write tcp 139.59.144.22:32942->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32942->104.36.82.14:4223: use of closed network connection Jul 26 22:03:45.818003280 (58h28m17.008301891s) [noise=true , address=104.36.82.6:4876 , piece_id=EMSUE4AA33FPR2G34KGMEVOOHUIRFAY2RQZQFYY3CZEDOS45AGEQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:51970->104.36.82.6:4876: use of closed network connection; write tcp 139.59.144.22:51970->104.36.82.6:4876: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51970->104.36.82.6:4876: use of closed network connection Jul 26 22:03:45.818224039 (58h28m17.008522654s) [seg_pos={0 0}, upload=56074]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:03:45.923329650 (58h28m17.113628222s) [upload=56074]: Done uploading segments... Jul 26 22:03:45.923360122 (58h28m17.113658692s) [upload=56074]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 15.246188114s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.337500924s FAIL (2025-07-26 22:04:05 UTC) orbiter warning: context deadline exceeded Duration: 6.337523917s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [............................................X.X..X..XXX....X] duration: 30.10877599s FAIL (2025-07-26 22:03:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== (58h27m19.160852834s) [upload=56059]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.970560064 (58h27m19.160858629s) [upload=56059]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.117843903 (58h27m19.30814249s) [upload=56059]: Done uploading segments... Jul 26 22:02:48.117867719 (58h27m19.308166287s) [upload=56059]: Upload finished. err: <nil> Jul 26 22:03:16.088194620 (58h27m47.278493235s) [upload=56074]: Starting upload Jul 26 22:03:16.088376905 (58h27m47.27867547s) [upload=56074]: Done starting upload Jul 26 22:03:16.088810540 (58h27m47.279109118s) [upload=56074]: Uploading segments... Jul 26 22:03:16.088950032 (58h27m47.279248621s) [upload=56074]: Got next segment. Inline: false Jul 26 22:03:16.088997348 (58h27m47.279295935s) [upload=56074]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:03:16.089003698 (58h27m47.279302287s) [upload=56074]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:03:16.235927985 (58h27m47.426226563s) [seg_pos={0 0}, upload=56074]: Begin upload segment... Jul 26 22:03:16.236008106 (58h27m47.426306692s) [seg_pos={0 0}, upload=56074]: Done begin upload segment. Jul 26 22:03:16.236047175 (58h27m47.426345761s) [noise=true , address=45.91.116.9:4263 , piece_id=GMZLI6VL3E646LGYVEWJ4H6EHKVD6EPETQGNBHCX6GFW4EREUMAQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.236554619 (58h27m47.426853198s) [noise=true , address=45.91.116.6:4973 , piece_id=7RQGHXVEUQIEUZM7DNSCID4CH4CDQZWIRZDYEUFET7LBATT7BPTA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.237546455 (58h27m47.427845037s) [noise=true , address=45.91.116.14:4189 , piece_id=ZCN7AL4H7P7MFIFBK6674FOHFDVOR7F2W4CIUHDL2RUJGDEJNISQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.237726616 (58h27m47.428025194s) [noise=true , address=45.91.116.12:4225 , piece_id=CTHWSYJ4NXGTNGA7DEXX2WHCGDXHAPXCMHVBWUJ6PYGN35MN4I5A, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.239644947 (58h27m47.429943535s) [noise=true , address=45.91.116.10:4082 , piece_id=EZYC47CADGVT2QTYCFJRJGQBVBDK5MK2SONEHCOHYOV43DLJS4GA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.240347271 (58h27m47.430645857s) [noise=true , address=45.91.116.13:4154 , piece_id=S6TYBP5H475KQQAE7RXJLG27BOXIFIRDN3DXLD6GZVFOVYZH46QA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.241494652 (58h27m47.43179324s) [noise=true , address=77.90.31.16:4734 , piece_id=BEOIIBUQVSKBMMIWRUDH3FNMTGDX5H4S2TZUPXEPMEAVF5HWAO3Q, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.242192055 (58h27m47.432490637s) [noise=true , address=104.36.82.3:4765 , piece_id=7F3B5K6MJZB7MZODDN3VSUUOWOREFLKNRHVZC26RWFCSF7P2MXJQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.244691319 (58h27m47.434989909s) [noise=true , address=104.36.82.4:4801 , piece_id=2COQU4P4L7DY2LUPVCTH2DLPCBYUQRKT6PYQ3BB2AVKAZWSWMQOA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.246470562 (58h27m47.436769151s) [noise=true , address=104.36.82.14:4223 , piece_id=GQMFBCVPMPZ7EMHJGNZKBWDZOYMVVIEYFXHM3QSH3O2OCKO7G2BA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.247902260 (58h27m47.438200851s) [noise=true , address=77.90.31.19:4841 , piece_id=BOF2X6NFK5FC5HVAYM7VZV7TELJEOZRWD4GRQPXNGGS32VDTNCIQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.254607394 (58h27m47.444905968s) [noise=true , address=104.36.82.6:4876 , piece_id=EMSUE4AA33FPR2G34KGMEVOOHUIRFAY2RQZQFYY3CZEDOS45AGEQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.268402565 (58h27m47.458701139s) [noise=true , address=45.91.116.11:4118 , piece_id=4RSJU3B3XNBJR3L6E673J4EFGIXRE2Y3PGB24ZKMIU3H5E3JIPUA, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.268822159 (58h27m47.459120733s) [noise=true , address=104.36.82.12:4150 , piece_id=5ZIOMTZQBC66A2SRYPO2MHKK6I4S2DCOFQTUSO5K3A5XCYRNSF3Q, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.270290736 (58h27m47.460589309s) [noise=true , address=45.91.116.7:4011 , piece_id=325G6VS4EHSDX7WXXNV3AMJBDM2FTEUATRCGCL7U7WOO5T66S6XQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.270873178 (58h27m47.461171749s) [noise=true , address=45.91.116.5:4299 , piece_id=4Y5AHJVCCP4VWJ4N5PY2WE4DSFLBJVZN3JPYC3V6JVTF6QCPNNZQ, seg_pos={0 0}, upload=56074]: Uploading piece... Jul 26 22:03:16.872897695 (58h27m48.063196287s) [upload=56074]: Next returned nil segment Jul 26 22:03:16.872968746 (58h27m48.063267334s) [upload=56074]: Waiting for error group managing segments... Jul 26 22:03:17.007002942 (58h27m48.197301515s) [noise=true , address=45.91.116.14:4189 , piece_id=ZCN7AL4H7P7MFIFBK6674FOHFDVOR7F2W4CIUHDL2RUJGDEJNISQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.007209108 (58h27m48.1975077s) [noise=true , address=45.91.116.10:4082 , piece_id=EZYC47CADGVT2QTYCFJRJGQBVBDK5MK2SONEHCOHYOV43DLJS4GA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.009322862 (58h27m48.19962144s) [noise=true , address=45.91.116.13:4154 , piece_id=S6TYBP5H475KQQAE7RXJLG27BOXIFIRDN3DXLD6GZVFOVYZH46QA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.009602813 (58h27m48.199901394s) [noise=true , address=45.91.116.6:4973 , piece_id=7RQGHXVEUQIEUZM7DNSCID4CH4CDQZWIRZDYEUFET7LBATT7BPTA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.015688237 (58h27m48.205986822s) [noise=true , address=45.91.116.12:4225 , piece_id=CTHWSYJ4NXGTNGA7DEXX2WHCGDXHAPXCMHVBWUJ6PYGN35MN4I5A, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.017443592 (58h27m48.207742183s) [noise=true , address=45.91.116.7:4011 , piece_id=325G6VS4EHSDX7WXXNV3AMJBDM2FTEUATRCGCL7U7WOO5T66S6XQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.019010997 (58h27m48.209309566s) [noise=true , address=45.91.116.5:4299 , piece_id=4Y5AHJVCCP4VWJ4N5PY2WE4DSFLBJVZN3JPYC3V6JVTF6QCPNNZQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.035401373 (58h27m48.225699938s) [noise=true , address=45.91.116.11:4118 , piece_id=4RSJU3B3XNBJR3L6E673J4EFGIXRE2Y3PGB24ZKMIU3H5E3JIPUA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:17.059860273 (58h27m48.250158857s) [noise=true , address=45.91.116.9:4263 , piece_id=GMZLI6VL3E646LGYVEWJ4H6EHKVD6EPETQGNBHCX6GFW4EREUMAQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:31.769542300 (58h28m2.959840889s) [noise=true , address=77.90.31.19:4841 , piece_id=BOF2X6NFK5FC5HVAYM7VZV7TELJEOZRWD4GRQPXNGGS32VDTNCIQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:42.488922757 (58h28m13.679221324s) [noise=true , address=104.36.82.3:4765 , piece_id=7F3B5K6MJZB7MZODDN3VSUUOWOREFLKNRHVZC26RWFCSF7P2MXJQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:44.654593132 (58h28m15.844891709s) [noise=true , address=77.90.31.16:4734 , piece_id=BEOIIBUQVSKBMMIWRUDH3FNMTGDX5H4S2TZUPXEPMEAVF5HWAO3Q, seg_pos={0 0}, upload=56074]: Done uploading piece. err: <nil> Jul 26 22:03:45.817542617 (58h28m17.007841197s) [noise=true , address=104.36.82.12:4150 , piece_id=5ZIOMTZQBC66A2SRYPO2MHKK6I4S2DCOFQTUSO5K3A5XCYRNSF3Q, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:104.36.82.12:4150): protocol: write tcp 139.59.144.22:50086->104.36.82.12:4150: use of closed network connection; write tcp 139.59.144.22:50086->104.36.82.12:4150: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50086->104.36.82.12:4150: use of closed network connection Jul 26 22:03:45.817841045 (58h28m17.008139634s) [noise=true , address=104.36.82.4:4801 , piece_id=2COQU4P4L7DY2LUPVCTH2DLPCBYUQRKT6PYQ3BB2AVKAZWSWMQOA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:42104->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:42104->104.36.82.4:4801: use of closed network connection Jul 26 22:03:45.817962004 (58h28m17.008260576s) [noise=true , address=104.36.82.14:4223 , piece_id=GQMFBCVPMPZ7EMHJGNZKBWDZOYMVVIEYFXHM3QSH3O2OCKO7G2BA, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:32942->104.36.82.14:4223: use of closed network connection; write tcp 139.59.144.22:32942->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:32942->104.36.82.14:4223: use of closed network connection Jul 26 22:03:45.818003280 (58h28m17.008301891s) [noise=true , address=104.36.82.6:4876 , piece_id=EMSUE4AA33FPR2G34KGMEVOOHUIRFAY2RQZQFYY3CZEDOS45AGEQ, seg_pos={0 0}, upload=56074]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:104.36.82.6:4876): protocol: write tcp 139.59.144.22:51970->104.36.82.6:4876: use of closed network connection; write tcp 139.59.144.22:51970->104.36.82.6:4876: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51970->104.36.82.6:4876: use of closed network connection Jul 26 22:03:45.818224039 (58h28m17.008522654s) [seg_pos={0 0}, upload=56074]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 22:03:45.923329650 (58h28m17.113628222s) [upload=56074]: Done uploading segments... Jul 26 22:03:45.923360122 (58h28m17.113658692s) [upload=56074]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.108844894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 49.493409252s FAIL (2025-07-26 22:03:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 49.493434518s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................XX.X..................] duration: 8.112468324s FAIL (2025-07-26 22:03:34 UTC) orbiter warning: uplink: context deadline exceeded Duration: 8.112504295s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 8.19426379s FAIL (2025-07-26 22:03:09 UTC) orbiter warning: context deadline exceeded Duration: 8.194289997s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.00383059s FAIL (2025-07-26 22:02:52 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== pload finished. err: <nil> Jul 26 22:01:52.714591975 (58h26m23.904890571s) [upload=56052]: Starting upload Jul 26 22:01:52.714687405 (58h26m23.904985971s) [upload=56052]: Done starting upload Jul 26 22:01:52.715003308 (58h26m23.905301895s) [upload=56052]: Uploading segments... Jul 26 22:01:52.715118502 (58h26m23.905417091s) [upload=56052]: Got next segment. Inline: false Jul 26 22:01:52.715172842 (58h26m23.905471431s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 22:01:52.715187853 (58h26m23.905486435s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 22:01:52.917108989 (58h26m24.10740757s) [seg_pos={0 0}, upload=56052]: Begin upload segment... Jul 26 22:01:52.917224901 (58h26m24.107523488s) [seg_pos={0 0}, upload=56052]: Done begin upload segment. Jul 26 22:01:52.917253801 (58h26m24.107552388s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917267374 (58h26m24.107565966s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917769918 (58h26m24.108068511s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.917964662 (58h26m24.108263243s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918227371 (58h26m24.108525963s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918468732 (58h26m24.108767319s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.918590544 (58h26m24.108889136s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.919051464 (58h26m24.109350068s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.920921977 (58h26m24.111220571s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921342741 (58h26m24.111641347s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.921816664 (58h26m24.112115256s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922249243 (58h26m24.112547835s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922631045 (58h26m24.112929638s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.922992197 (58h26m24.113290803s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923375550 (58h26m24.113674147s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.923751211 (58h26m24.114049806s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924228209 (58h26m24.114526801s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:52.924589445 (58h26m24.114888013s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Uploading piece... Jul 26 22:01:53.550905014 (58h26m24.741203593s) [upload=56052]: Next returned nil segment Jul 26 22:01:53.550927130 (58h26m24.74122571s) [upload=56052]: Waiting for error group managing segments... Jul 26 22:01:53.819407215 (58h26m25.009705804s) [noise=true , address=79.127.219.38:4005 , piece_id=5MBTBO2HOW3KSSDUCAOXHMKQ5KUF5PJ6ZVHZL2YIFLH2HFBGKOCA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.346461178 (58h26m25.536759752s) [noise=true , address=79.127.205.245:4239 , piece_id=SGENN2KU5HKHIJD5GHJ7SFLFFI26SX24CUUEDSY3B3NFQ6T6FRMQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:54.611657044 (58h26m25.801955609s) [noise=true , address=79.127.205.246:4280 , piece_id=HSSMHMMISWIA3RU7MBR77JWHZWK2I2VWBSDPIY5UTHSRMOTIHISQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:01:59.487454052 (58h26m30.677752631s) [noise=true , address=45.91.116.14:4206 , piece_id=4MGRTWXEEP6AXES2HYD3C36OIMBZYFMUQLZVVK75HTRJHZ23NP2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:01.120823839 (58h26m32.311122412s) [noise=true , address=79.127.201.215:4359 , piece_id=MJJVECJBANIFMSU5S5IGLGPWXCQXD3QYI2A2AAVLQRPO7AYEKQWQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.098729050 (58h26m33.289027638s) [noise=true , address=77.90.31.17:4780 , piece_id=274UDBDAM5OSFMCBJHYME4VAS524HSYVOPRJS6G7BOHMFTIJWLPQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:02.519193611 (58h26m33.709492181s) [noise=true , address=77.90.31.13:4656 , piece_id=HUASUVWDFUEPDZDNMU7DU6U3DHTKIVIBDF4UXRWNRXMIWHCKMALQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.074616386 (58h26m39.264914967s) [noise=true , address=104.36.82.44:4876 , piece_id=SPGU4PBFIIAYUNFL4ZQZQVCPOYV2RHMBOZ7UH5IYZWYARY2GZX2Q, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:08.824226469 (58h26m40.014525052s) [noise=true , address=104.36.82.38:4695 , piece_id=VQ53AIN5HO2ZQFCN54DP2XDCMCW6SL73RCFHIWXCDSDOIXULJHEA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:09.725756719 (58h26m40.916055301s) [noise=true , address=104.36.82.55:4204 , piece_id=CTXBNL4IZ3R3OBARIDGOY34SU225U4L5FZCATXW7JTLNJKC4V4ZA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:10.840805673 (58h26m42.031104241s) [noise=true , address=104.36.82.7:4933 , piece_id=WKEA3AKAJSETP2FGCLWYFV3IMXDXFUBJOG7CD7WDLIGO4EUZB25A, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:16.569789663 (58h26m47.760088235s) [noise=true , address=104.36.82.53:4123 , piece_id=EAXK6QNCYUUXJ5T7E4NQVCUHOE3UEM3MM34PTUMW4M6Y3BGIZ6TQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:46.232587575 (58h27m17.422886162s) [noise=true , address=104.36.82.45:4929 , piece_id=A7IDHL7AUYQVIEUS6DEZB6EWRB43AEVUPEZYJ7SIWQFAGFYYV3AQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957885529 (58h27m19.148184101s) [noise=true , address=104.36.82.5:4851 , piece_id=J7XOJCDQHWDCMAKOI2HJVVD2LQ4LUWCHXDRSD2O54AHLXQUR3JDA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: <nil> Jul 26 22:02:47.957948293 (58h27m19.148246875s) [seg_pos={0 0}, upload=56052]: Segment reached optimal threshold of 14 pieces. Jul 26 22:02:47.958142403 (58h27m19.148440972s) [noise=true , address=104.36.82.47:4519 , piece_id=2RXZNJS5V4S62RVSKXL3IS4ZJTGYS66Q43ZSYTFNRCMZ4PKMLJSQ, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12enbzhEQp1y2H8Fwdb5UqxcMDdgd5HYb5jYfo2U6Fnycvscaw3): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958260486 (58h27m19.148559058s) [noise=true , address=104.36.82.52:4110 , piece_id=AKBKWDX4DZU7C5WWNL4LIOELH4OIDLT7KI5HSNJKNLTISWLTSHUA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftGsmDR7RLZnGZzHLnE7g4PSTkpBo4JLN5MBaXPjPciPETYui): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958337983 (58h27m19.148636552s) [noise=true , address=104.36.82.20:4450 , piece_id=E5Q5TCTWQVTSPJXCRF2R5JPYQCZU4UR5BELPZ3ND3ZTRRVMKXORA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UeL1LySKQySa8wjvG4FBs69cE1iWkW4Diz83nu1PxBCKxyvki): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958388008 (58h27m19.148686581s) [noise=true , address=104.36.82.46:4488 , piece_id=3RBVOL4SFSCJS4KIOWOF7MWWDVQQMZIYAU7RBTE2B4BWMC5EEEIA, seg_pos={0 0}, upload=56052]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MU3fdv9cTkq4pUZ3h989STAG1wxrbhFgzmr6tcrfumZKHP9BZ): protocol: expected piece hash; context canceled; EOF Jul 26 22:02:47.958412723 (58h27m19.148711302s) [seg_pos={0 0}, upload=56052]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 26 22:02:47.958517135 (58h27m19.148815718s) [upload=56052]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 22:02:47.958524514 (58h27m19.1488231s) [upload=56052]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 22:02:48.115659888 (58h27m19.30595846s) [upload=56052]: Done uploading segments... Jul 26 22:02:48.115691908 (58h27m19.30599048s) [upload=56052]: Upload finished. err: <nil> Duration: 1m0.003842732s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................................X.X..X..XXX....] duration: 29.056840518s FAIL (2025-07-26 22:02:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12.841872124 (58h23m44.032170692s) [seg_pos={0 0}, upload=55998]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:59:12.841975383 (58h23m44.032273954s) [upload=55998]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:59:12.841981090 (58h23m44.032279669s) [upload=55998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:59:13.007332906 (58h23m44.197631473s) [upload=55998]: Done uploading segments... Jul 26 21:59:13.007353428 (58h23m44.197651993s) [upload=55998]: Upload finished. err: <nil> Jul 26 21:59:29.947865209 (58h24m1.138163778s) [upload=56010]: Starting upload Jul 26 21:59:29.947935985 (58h24m1.138234567s) [upload=56010]: Done starting upload Jul 26 21:59:29.948149160 (58h24m1.138447746s) [upload=56010]: Uploading segments... Jul 26 21:59:29.948467840 (58h24m1.13876643s) [upload=56010]: Got next segment. Inline: false Jul 26 21:59:29.948543796 (58h24m1.138842387s) [upload=56010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:59:29.948554079 (58h24m1.138852669s) [upload=56010]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:59:30.054416403 (58h24m1.244715051s) [seg_pos={0 0}, upload=56010]: Begin upload segment... Jul 26 21:59:30.054482164 (58h24m1.244780736s) [seg_pos={0 0}, upload=56010]: Done begin upload segment. Jul 26 21:59:30.054510509 (58h24m1.244809156s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055167221 (58h24m1.245465801s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055644524 (58h24m1.245943104s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.056343685 (58h24m1.246642349s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.057586727 (58h24m1.247885323s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058142658 (58h24m1.248441246s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058734112 (58h24m1.249032766s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059227272 (58h24m1.249525863s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059619036 (58h24m1.249917607s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.060591780 (58h24m1.250890372s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061080040 (58h24m1.251378632s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061492487 (58h24m1.251791073s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.063080062 (58h24m1.253378649s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.064540940 (58h24m1.254839527s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.065810124 (58h24m1.256108713s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.066469941 (58h24m1.256768549s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.661489882 (58h24m1.851788448s) [upload=56010]: Next returned nil segment Jul 26 21:59:30.661508940 (58h24m1.851807522s) [upload=56010]: Waiting for error group managing segments... Jul 26 21:59:30.800995473 (58h24m1.991294053s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804285922 (58h24m1.994584514s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804429337 (58h24m1.994727921s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.808253663 (58h24m1.998552268s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.813099424 (58h24m2.00339799s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.814982618 (58h24m2.005281186s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.815752512 (58h24m2.0060511s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833119899 (58h24m2.023418467s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833220827 (58h24m2.023519393s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.842324478 (58h24m2.032623046s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:40.421523005 (58h24m11.611821581s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:50.748021926 (58h24m21.93832051s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:59.677978359 (58h24m30.868276941s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection Jul 26 21:59:59.678014748 (58h24m30.868313314s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:59:59.678107054 (58h24m30.868405637s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection Jul 26 21:59:59.678142898 (58h24m30.868441481s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection Jul 26 21:59:59.678180866 (58h24m30.868479449s) [seg_pos={0 0}, upload=56010]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:59:59.784815051 (58h24m30.975113634s) [upload=56010]: Done uploading segments... Jul 26 21:59:59.784852928 (58h24m30.975151509s) [upload=56010]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 29.056977242s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.337113887s FAIL (2025-07-26 22:02:33 UTC) orbiter warning: context deadline exceeded Duration: 5.337141711s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................XX.X.................] duration: 8.980764991s FAIL (2025-07-26 22:02:24 UTC) orbiter warning: uplink: context deadline exceeded Duration: 8.980856583s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................XX.X................] duration: 23.096616208s FAIL (2025-07-26 22:02:15 UTC) orbiter warning: uplink: context deadline exceeded Duration: 23.096655996s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 28.762421912s FAIL (2025-07-26 22:01:54 UTC) orbiter warning: context deadline exceeded Duration: 28.762459146s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................................X.X..X..XXX...] duration: 2.316141301s FAIL (2025-07-26 22:01:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12.841872124 (58h23m44.032170692s) [seg_pos={0 0}, upload=55998]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:59:12.841975383 (58h23m44.032273954s) [upload=55998]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:59:12.841981090 (58h23m44.032279669s) [upload=55998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:59:13.007332906 (58h23m44.197631473s) [upload=55998]: Done uploading segments... Jul 26 21:59:13.007353428 (58h23m44.197651993s) [upload=55998]: Upload finished. err: <nil> Jul 26 21:59:29.947865209 (58h24m1.138163778s) [upload=56010]: Starting upload Jul 26 21:59:29.947935985 (58h24m1.138234567s) [upload=56010]: Done starting upload Jul 26 21:59:29.948149160 (58h24m1.138447746s) [upload=56010]: Uploading segments... Jul 26 21:59:29.948467840 (58h24m1.13876643s) [upload=56010]: Got next segment. Inline: false Jul 26 21:59:29.948543796 (58h24m1.138842387s) [upload=56010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:59:29.948554079 (58h24m1.138852669s) [upload=56010]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:59:30.054416403 (58h24m1.244715051s) [seg_pos={0 0}, upload=56010]: Begin upload segment... Jul 26 21:59:30.054482164 (58h24m1.244780736s) [seg_pos={0 0}, upload=56010]: Done begin upload segment. Jul 26 21:59:30.054510509 (58h24m1.244809156s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055167221 (58h24m1.245465801s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055644524 (58h24m1.245943104s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.056343685 (58h24m1.246642349s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.057586727 (58h24m1.247885323s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058142658 (58h24m1.248441246s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058734112 (58h24m1.249032766s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059227272 (58h24m1.249525863s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059619036 (58h24m1.249917607s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.060591780 (58h24m1.250890372s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061080040 (58h24m1.251378632s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061492487 (58h24m1.251791073s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.063080062 (58h24m1.253378649s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.064540940 (58h24m1.254839527s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.065810124 (58h24m1.256108713s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.066469941 (58h24m1.256768549s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.661489882 (58h24m1.851788448s) [upload=56010]: Next returned nil segment Jul 26 21:59:30.661508940 (58h24m1.851807522s) [upload=56010]: Waiting for error group managing segments... Jul 26 21:59:30.800995473 (58h24m1.991294053s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804285922 (58h24m1.994584514s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804429337 (58h24m1.994727921s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.808253663 (58h24m1.998552268s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.813099424 (58h24m2.00339799s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.814982618 (58h24m2.005281186s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.815752512 (58h24m2.0060511s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833119899 (58h24m2.023418467s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833220827 (58h24m2.023519393s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.842324478 (58h24m2.032623046s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:40.421523005 (58h24m11.611821581s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:50.748021926 (58h24m21.93832051s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:59.677978359 (58h24m30.868276941s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection Jul 26 21:59:59.678014748 (58h24m30.868313314s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:59:59.678107054 (58h24m30.868405637s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection Jul 26 21:59:59.678142898 (58h24m30.868441481s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection Jul 26 21:59:59.678180866 (58h24m30.868479449s) [seg_pos={0 0}, upload=56010]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:59:59.784815051 (58h24m30.975113634s) [upload=56010]: Done uploading segments... Jul 26 21:59:59.784852928 (58h24m30.975151509s) [upload=56010]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.316167329s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................................X.X..X..XXX..] duration: 30.002291122s FAIL (2025-07-26 22:01:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12.841872124 (58h23m44.032170692s) [seg_pos={0 0}, upload=55998]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:59:12.841975383 (58h23m44.032273954s) [upload=55998]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:59:12.841981090 (58h23m44.032279669s) [upload=55998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:59:13.007332906 (58h23m44.197631473s) [upload=55998]: Done uploading segments... Jul 26 21:59:13.007353428 (58h23m44.197651993s) [upload=55998]: Upload finished. err: <nil> Jul 26 21:59:29.947865209 (58h24m1.138163778s) [upload=56010]: Starting upload Jul 26 21:59:29.947935985 (58h24m1.138234567s) [upload=56010]: Done starting upload Jul 26 21:59:29.948149160 (58h24m1.138447746s) [upload=56010]: Uploading segments... Jul 26 21:59:29.948467840 (58h24m1.13876643s) [upload=56010]: Got next segment. Inline: false Jul 26 21:59:29.948543796 (58h24m1.138842387s) [upload=56010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:59:29.948554079 (58h24m1.138852669s) [upload=56010]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:59:30.054416403 (58h24m1.244715051s) [seg_pos={0 0}, upload=56010]: Begin upload segment... Jul 26 21:59:30.054482164 (58h24m1.244780736s) [seg_pos={0 0}, upload=56010]: Done begin upload segment. Jul 26 21:59:30.054510509 (58h24m1.244809156s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055167221 (58h24m1.245465801s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055644524 (58h24m1.245943104s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.056343685 (58h24m1.246642349s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.057586727 (58h24m1.247885323s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058142658 (58h24m1.248441246s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058734112 (58h24m1.249032766s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059227272 (58h24m1.249525863s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059619036 (58h24m1.249917607s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.060591780 (58h24m1.250890372s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061080040 (58h24m1.251378632s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061492487 (58h24m1.251791073s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.063080062 (58h24m1.253378649s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.064540940 (58h24m1.254839527s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.065810124 (58h24m1.256108713s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.066469941 (58h24m1.256768549s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.661489882 (58h24m1.851788448s) [upload=56010]: Next returned nil segment Jul 26 21:59:30.661508940 (58h24m1.851807522s) [upload=56010]: Waiting for error group managing segments... Jul 26 21:59:30.800995473 (58h24m1.991294053s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804285922 (58h24m1.994584514s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804429337 (58h24m1.994727921s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.808253663 (58h24m1.998552268s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.813099424 (58h24m2.00339799s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.814982618 (58h24m2.005281186s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.815752512 (58h24m2.0060511s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833119899 (58h24m2.023418467s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833220827 (58h24m2.023519393s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.842324478 (58h24m2.032623046s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:40.421523005 (58h24m11.611821581s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:50.748021926 (58h24m21.93832051s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:59.677978359 (58h24m30.868276941s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection Jul 26 21:59:59.678014748 (58h24m30.868313314s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:59:59.678107054 (58h24m30.868405637s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection Jul 26 21:59:59.678142898 (58h24m30.868441481s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection Jul 26 21:59:59.678180866 (58h24m30.868479449s) [seg_pos={0 0}, upload=56010]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:59:59.784815051 (58h24m30.975113634s) [upload=56010]: Done uploading segments... Jul 26 21:59:59.784852928 (58h24m30.975151509s) [upload=56010]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.002325501s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.773449993s FAIL (2025-07-26 22:01:16 UTC) orbiter warning: context deadline exceeded Duration: 5.773494316s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XX.X...............] duration: 22.592866223s FAIL (2025-07-26 22:01:02 UTC) orbiter warning: uplink: context deadline exceeded Duration: 22.592995552s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.647489868s FAIL (2025-07-26 22:00:45 UTC) orbiter warning: context deadline exceeded Duration: 2.647538808s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................X.X..X..XXX.] duration: 27.031545626s FAIL (2025-07-26 22:00:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12.841872124 (58h23m44.032170692s) [seg_pos={0 0}, upload=55998]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:59:12.841975383 (58h23m44.032273954s) [upload=55998]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:59:12.841981090 (58h23m44.032279669s) [upload=55998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:59:13.007332906 (58h23m44.197631473s) [upload=55998]: Done uploading segments... Jul 26 21:59:13.007353428 (58h23m44.197651993s) [upload=55998]: Upload finished. err: <nil> Jul 26 21:59:29.947865209 (58h24m1.138163778s) [upload=56010]: Starting upload Jul 26 21:59:29.947935985 (58h24m1.138234567s) [upload=56010]: Done starting upload Jul 26 21:59:29.948149160 (58h24m1.138447746s) [upload=56010]: Uploading segments... Jul 26 21:59:29.948467840 (58h24m1.13876643s) [upload=56010]: Got next segment. Inline: false Jul 26 21:59:29.948543796 (58h24m1.138842387s) [upload=56010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:59:29.948554079 (58h24m1.138852669s) [upload=56010]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:59:30.054416403 (58h24m1.244715051s) [seg_pos={0 0}, upload=56010]: Begin upload segment... Jul 26 21:59:30.054482164 (58h24m1.244780736s) [seg_pos={0 0}, upload=56010]: Done begin upload segment. Jul 26 21:59:30.054510509 (58h24m1.244809156s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055167221 (58h24m1.245465801s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055644524 (58h24m1.245943104s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.056343685 (58h24m1.246642349s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.057586727 (58h24m1.247885323s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058142658 (58h24m1.248441246s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058734112 (58h24m1.249032766s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059227272 (58h24m1.249525863s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059619036 (58h24m1.249917607s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.060591780 (58h24m1.250890372s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061080040 (58h24m1.251378632s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061492487 (58h24m1.251791073s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.063080062 (58h24m1.253378649s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.064540940 (58h24m1.254839527s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.065810124 (58h24m1.256108713s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.066469941 (58h24m1.256768549s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.661489882 (58h24m1.851788448s) [upload=56010]: Next returned nil segment Jul 26 21:59:30.661508940 (58h24m1.851807522s) [upload=56010]: Waiting for error group managing segments... Jul 26 21:59:30.800995473 (58h24m1.991294053s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804285922 (58h24m1.994584514s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804429337 (58h24m1.994727921s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.808253663 (58h24m1.998552268s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.813099424 (58h24m2.00339799s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.814982618 (58h24m2.005281186s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.815752512 (58h24m2.0060511s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833119899 (58h24m2.023418467s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833220827 (58h24m2.023519393s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.842324478 (58h24m2.032623046s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:40.421523005 (58h24m11.611821581s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:50.748021926 (58h24m21.93832051s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:59.677978359 (58h24m30.868276941s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection Jul 26 21:59:59.678014748 (58h24m30.868313314s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:59:59.678107054 (58h24m30.868405637s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection Jul 26 21:59:59.678142898 (58h24m30.868441481s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection Jul 26 21:59:59.678180866 (58h24m30.868479449s) [seg_pos={0 0}, upload=56010]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:59:59.784815051 (58h24m30.975113634s) [upload=56010]: Done uploading segments... Jul 26 21:59:59.784852928 (58h24m30.975151509s) [upload=56010]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 27.031573194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XX.X..............] duration: 25.794381362s FAIL (2025-07-26 22:00:17 UTC) orbiter warning: uplink: context deadline exceeded Duration: 25.794420208s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 11.33182707s FAIL (2025-07-26 22:00:16 UTC) orbiter warning: context deadline exceeded Duration: 11.331867898s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................................X.X..X..XXX] duration: 30.109366877s FAIL (2025-07-26 21:59:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 12.841872124 (58h23m44.032170692s) [seg_pos={0 0}, upload=55998]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:59:12.841975383 (58h23m44.032273954s) [upload=55998]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:59:12.841981090 (58h23m44.032279669s) [upload=55998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:59:13.007332906 (58h23m44.197631473s) [upload=55998]: Done uploading segments... Jul 26 21:59:13.007353428 (58h23m44.197651993s) [upload=55998]: Upload finished. err: <nil> Jul 26 21:59:29.947865209 (58h24m1.138163778s) [upload=56010]: Starting upload Jul 26 21:59:29.947935985 (58h24m1.138234567s) [upload=56010]: Done starting upload Jul 26 21:59:29.948149160 (58h24m1.138447746s) [upload=56010]: Uploading segments... Jul 26 21:59:29.948467840 (58h24m1.13876643s) [upload=56010]: Got next segment. Inline: false Jul 26 21:59:29.948543796 (58h24m1.138842387s) [upload=56010]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:59:29.948554079 (58h24m1.138852669s) [upload=56010]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:59:30.054416403 (58h24m1.244715051s) [seg_pos={0 0}, upload=56010]: Begin upload segment... Jul 26 21:59:30.054482164 (58h24m1.244780736s) [seg_pos={0 0}, upload=56010]: Done begin upload segment. Jul 26 21:59:30.054510509 (58h24m1.244809156s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055167221 (58h24m1.245465801s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.055644524 (58h24m1.245943104s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.056343685 (58h24m1.246642349s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.057586727 (58h24m1.247885323s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058142658 (58h24m1.248441246s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.058734112 (58h24m1.249032766s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059227272 (58h24m1.249525863s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.059619036 (58h24m1.249917607s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.060591780 (58h24m1.250890372s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061080040 (58h24m1.251378632s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.061492487 (58h24m1.251791073s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.063080062 (58h24m1.253378649s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.064540940 (58h24m1.254839527s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.065810124 (58h24m1.256108713s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.066469941 (58h24m1.256768549s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Uploading piece... Jul 26 21:59:30.661489882 (58h24m1.851788448s) [upload=56010]: Next returned nil segment Jul 26 21:59:30.661508940 (58h24m1.851807522s) [upload=56010]: Waiting for error group managing segments... Jul 26 21:59:30.800995473 (58h24m1.991294053s) [noise=true , address=45.91.116.11:4119 , piece_id=56HDMQ7523WPT3A45JX4AT3PHHPGGGAMUXTV3UWVC5BZZO7X5TFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804285922 (58h24m1.994584514s) [noise=true , address=45.91.116.13:4154 , piece_id=FN7NMXGBSMU2MU3JVUZ2YC4X4QJEZ6YWFVMB77275UHVWY5DRAXQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.804429337 (58h24m1.994727921s) [noise=true , address=45.91.116.5:4297 , piece_id=P6ASHYRKN6UK2WNSJ72GVZGMC35OOAHIVAJBXYQIEECK52G6DYAA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.808253663 (58h24m1.998552268s) [noise=true , address=45.91.116.9:4262 , piece_id=BD2GJTDLH275YNTMJOKDSRUX2CGXC2UT3YTPMK6M6WBOPMF6KDWQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.813099424 (58h24m2.00339799s) [noise=true , address=45.91.116.13:4155 , piece_id=6ZHSLLRCXSHNCMFGBJJEJE6TFPLLRNGWOBPI6QAOLWYJYQXCHK6A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.814982618 (58h24m2.005281186s) [noise=true , address=45.91.116.4:4939 , piece_id=3YLUWP7OO65GYJP4RBX5VICKPH7JY7TWEKJP2ZVKZIPKR766SU2Q, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.815752512 (58h24m2.0060511s) [noise=true , address=45.91.116.12:4226 , piece_id=EENIKWKHONAAFPQY636DH4CTMJ5HBJIM2CMWWUTLI72JOEBANUSA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833119899 (58h24m2.023418467s) [noise=true , address=45.91.116.11:4118 , piece_id=VW4GH5QIK6EYDJLSGFZS3S3S7QO662WHU4OSDPX52YI5EWHFIZ2A, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.833220827 (58h24m2.023519393s) [noise=true , address=45.91.116.6:4974 , piece_id=XNEJHEMAIDFTYQZUOR4I7BVUDVLJR4KSU47NI7SZBNYN27K7VCGQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:30.842324478 (58h24m2.032623046s) [noise=true , address=45.91.116.10:4083 , piece_id=RD3PZ7D7WDK7RCIIU7ZLBP7OUXWUXLQEDSKGPWNIAXAWBDBQ3DZQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:40.421523005 (58h24m11.611821581s) [noise=true , address=77.90.31.14:4663 , piece_id=ZCFZNEQ7A7TU6RWP67JVABO2264X3C7S2AY6VP5APZTFM3QTCMPA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:50.748021926 (58h24m21.93832051s) [noise=true , address=104.36.82.13:4187 , piece_id=N2ALF4E4LJE42PZFFZBHF5DSGIUPXJ6AVVPWFD23C4BFHYJW5HQQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: <nil> Jul 26 21:59:59.677978359 (58h24m30.868276941s) [noise=true , address=77.90.31.11:4555 , piece_id=OLDBKXZKZKFXK6N4H2MH5IMP5BYESVNHUZQP4P36WHXKQ6DVSJOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50542->77.90.31.11:4555: use of closed network connection Jul 26 21:59:59.678014748 (58h24m30.868313314s) [noise=true , address=77.90.31.10:4519 , piece_id=4WSDJKSBAFEIEFXBA5JULAWTJKZEJBUUZM2N2UAFBAO3YL3QY3BA, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:59:59.678107054 (58h24m30.868405637s) [noise=true , address=104.36.82.8:4971 , piece_id=4MDDVMQZ6ASMEHP27MRBWYLEX2L4GCZSCDGWLEBNVTDIXUIH6HOQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:104.36.82.8:4971): protocol: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42660->104.36.82.8:4971: use of closed network connection Jul 26 21:59:59.678142898 (58h24m30.868441481s) [noise=true , address=104.36.82.3:4767 , piece_id=ZHBZIIDS3HB6SYMCOOVDXYT2N2MSRCALPMMRKOATEBY54CBH6MFQ, seg_pos={0 0}, upload=56010]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:51848->104.36.82.3:4767: use of closed network connection Jul 26 21:59:59.678180866 (58h24m30.868479449s) [seg_pos={0 0}, upload=56010]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:59:59.784815051 (58h24m30.975113634s) [upload=56010]: Done uploading segments... Jul 26 21:59:59.784852928 (58h24m30.975151509s) [upload=56010]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.109394318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................................X.X..X..XX] duration: 30.00418401s FAIL (2025-07-26 21:59:15 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== >104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:56618->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56618->104.36.82.4:4801: use of closed network connection Jul 26 21:58:25.197342845 (58h22m56.387641425s) [seg_pos={0 0}, upload=55984]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:58:25.197437160 (58h22m56.38773575s) [upload=55984]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:58:25.197443230 (58h22m56.387741816s) [upload=55984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:58:25.352579068 (58h22m56.542877652s) [upload=55984]: Done uploading segments... Jul 26 21:58:25.352624830 (58h22m56.542923402s) [upload=55984]: Upload finished. err: <nil> Jul 26 21:58:45.493859912 (58h23m16.684158492s) [upload=55998]: Starting upload Jul 26 21:58:45.493947312 (58h23m16.684245893s) [upload=55998]: Done starting upload Jul 26 21:58:45.494045566 (58h23m16.684344145s) [upload=55998]: Uploading segments... Jul 26 21:58:45.494241199 (58h23m16.684539765s) [upload=55998]: Got next segment. Inline: false Jul 26 21:58:45.494276989 (58h23m16.68457556s) [upload=55998]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:58:45.494283927 (58h23m16.684582507s) [upload=55998]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:58:45.691992584 (58h23m16.882291163s) [seg_pos={0 0}, upload=55998]: Begin upload segment... Jul 26 21:58:45.692194538 (58h23m16.882493124s) [noise=true , address=104.36.82.3:4765 , piece_id=VNIKPXZDDR46TNGXEAZMKWNNHQHNSTN53ZSDI3IHI4M6YT6H2S3Q, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.692233741 (58h23m16.882532328s) [noise=true , address=104.36.82.9:4005 , piece_id=PBARHKECSK6VLTITC3XP4DEXDME3MNMM6BK6S5IJWUSVEO6HJV2Q, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.692603419 (58h23m16.882901998s) [noise=true , address=45.91.116.5:4298 , piece_id=4IBQ3FA5IK3FMI2IUXS73KKSKVMVDMOO5KP4IQV44RVVVTKOALXA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.692718179 (58h23m16.883016759s) [noise=true , address=45.91.116.13:4154 , piece_id=QCLY2AL4MHREOJV26OCHZCTP7Z5FVQSQ7DJQBF7F2WINNOTZB2ZA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.692899201 (58h23m16.883197773s) [seg_pos={0 0}, upload=55998]: Done begin upload segment. Jul 26 21:58:45.693041076 (58h23m16.883339658s) [noise=true , address=45.91.116.12:4225 , piece_id=PLAAKSIFEVFS7YKQW3YEQMCTREWPZPLJT5PJYSRFY7WGJMXWWCXQ, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.693315237 (58h23m16.883613816s) [noise=true , address=45.91.116.14:4190 , piece_id=RP27OBJ43SR2236INZCQ2BGSOKXPN6XEOR72LZCBZ2REYBFDMGIQ, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.693689091 (58h23m16.883987673s) [noise=true , address=45.91.116.10:4082 , piece_id=5K3CG5KLABG62WXZBXKSCWMU4OSHERE3QODYHBCIHM7KQB2DNZVA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.694096016 (58h23m16.884394607s) [noise=true , address=104.36.82.7:4909 , piece_id=EYSWYXT6ZX3W2CYXQGYSOR7OG6JKTBBTUUW4RZORJDZMG7WS2O5A, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.694539274 (58h23m16.884837871s) [noise=true , address=77.90.31.19:4841 , piece_id=KATOV5RVYGA3TSVQTUO4BRTGRWBBTBXLFZV4BNJCITLR4PI6TYOA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.694953351 (58h23m16.885251934s) [noise=true , address=45.91.116.6:4974 , piece_id=7FMB2UUA2PWSP3LTP6WVDR34BQSGRLVDFRKS2H7TIMTO2MJLINNQ, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.695310064 (58h23m16.885608649s) [noise=true , address=45.91.116.8:4047 , piece_id=YP3MEMD3I3PHVZGUTIKNSO7QU7YUZMHVLTMLGPXZALR756QPEHLA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.695564954 (58h23m16.88586354s) [noise=true , address=104.36.82.14:4223 , piece_id=THSYZ7U4H4IV7K6UN4KHAO5IC52X4WR72GI6YVLJ2JOBX4RTSJDQ, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.695885467 (58h23m16.886184053s) [noise=true , address=45.91.116.13:4153 , piece_id=AEMIAVQIDGEJMSL5K73QWJSADX4LWWLPGJEOCIN5F4Z3YNNN5FZA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.696256254 (58h23m16.886554838s) [noise=true , address=45.91.116.7:4010 , piece_id=EP36JMCUA632MGL3EPIASO4IXQIE5VY4GGQEVTKYXUCH7F3GBUIQ, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.696518776 (58h23m16.88681736s) [noise=true , address=104.36.82.8:4971 , piece_id=SEKFGBUODVWQGM3SMIC2DAUEHGMG4NMNJOWOZTEQS3TN7LHB3EZA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:45.696853365 (58h23m16.887151951s) [noise=true , address=77.90.31.18:4807 , piece_id=U64LU75ELCYPHNIDSVEKZHIQNCA3737RMXUAOJ2S3LAD2DIHVLTA, seg_pos={0 0}, upload=55998]: Uploading piece... Jul 26 21:58:46.308937030 (58h23m17.499235613s) [upload=55998]: Next returned nil segment Jul 26 21:58:46.308955807 (58h23m17.499254397s) [upload=55998]: Waiting for error group managing segments... Jul 26 21:58:46.443657606 (58h23m17.633956177s) [noise=true , address=45.91.116.12:4225 , piece_id=PLAAKSIFEVFS7YKQW3YEQMCTREWPZPLJT5PJYSRFY7WGJMXWWCXQ, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.443976278 (58h23m17.634274872s) [noise=true , address=45.91.116.13:4154 , piece_id=QCLY2AL4MHREOJV26OCHZCTP7Z5FVQSQ7DJQBF7F2WINNOTZB2ZA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.446598460 (58h23m17.636897046s) [noise=true , address=45.91.116.7:4010 , piece_id=EP36JMCUA632MGL3EPIASO4IXQIE5VY4GGQEVTKYXUCH7F3GBUIQ, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.450117254 (58h23m17.640415842s) [noise=true , address=45.91.116.14:4190 , piece_id=RP27OBJ43SR2236INZCQ2BGSOKXPN6XEOR72LZCBZ2REYBFDMGIQ, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.450327022 (58h23m17.640625611s) [noise=true , address=45.91.116.10:4082 , piece_id=5K3CG5KLABG62WXZBXKSCWMU4OSHERE3QODYHBCIHM7KQB2DNZVA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.453735451 (58h23m17.644034042s) [noise=true , address=45.91.116.8:4047 , piece_id=YP3MEMD3I3PHVZGUTIKNSO7QU7YUZMHVLTMLGPXZALR756QPEHLA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.456997323 (58h23m17.64729591s) [noise=true , address=45.91.116.5:4298 , piece_id=4IBQ3FA5IK3FMI2IUXS73KKSKVMVDMOO5KP4IQV44RVVVTKOALXA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.459528125 (58h23m17.649826698s) [noise=true , address=45.91.116.6:4974 , piece_id=7FMB2UUA2PWSP3LTP6WVDR34BQSGRLVDFRKS2H7TIMTO2MJLINNQ, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:46.464213675 (58h23m17.654512265s) [noise=true , address=45.91.116.13:4153 , piece_id=AEMIAVQIDGEJMSL5K73QWJSADX4LWWLPGJEOCIN5F4Z3YNNN5FZA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:58:55.345877230 (58h23m26.536175802s) [noise=true , address=77.90.31.19:4841 , piece_id=KATOV5RVYGA3TSVQTUO4BRTGRWBBTBXLFZV4BNJCITLR4PI6TYOA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:59:04.668080703 (58h23m35.858379278s) [noise=true , address=104.36.82.8:4971 , piece_id=SEKFGBUODVWQGM3SMIC2DAUEHGMG4NMNJOWOZTEQS3TN7LHB3EZA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:59:06.338516968 (58h23m37.528815536s) [noise=true , address=104.36.82.9:4005 , piece_id=PBARHKECSK6VLTITC3XP4DEXDME3MNMM6BK6S5IJWUSVEO6HJV2Q, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:59:12.841465813 (58h23m44.031764384s) [noise=true , address=104.36.82.14:4223 , piece_id=THSYZ7U4H4IV7K6UN4KHAO5IC52X4WR72GI6YVLJ2JOBX4RTSJDQ, seg_pos={0 0}, upload=55998]: Done uploading piece. err: <nil> Jul 26 21:59:12.841499491 (58h23m44.03179807s) [seg_pos={0 0}, upload=55998]: Segment reached optimal threshold of 13 pieces. Jul 26 21:59:12.841669929 (58h23m44.031968498s) [noise=true , address=104.36.82.7:4909 , piece_id=EYSWYXT6ZX3W2CYXQGYSOR7OG6JKTBBTUUW4RZORJDZMG7WS2O5A, seg_pos={0 0}, upload=55998]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: expected piece hash; context canceled; EOF Jul 26 21:59:12.841735318 (58h23m44.032033888s) [noise=true , address=104.36.82.3:4765 , piece_id=VNIKPXZDDR46TNGXEAZMKWNNHQHNSTN53ZSDI3IHI4M6YT6H2S3Q, seg_pos={0 0}, upload=55998]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV): protocol: expected piece hash; context canceled; EOF Jul 26 21:59:12.841831154 (58h23m44.032129724s) [noise=true , address=77.90.31.18:4807 , piece_id=U64LU75ELCYPHNIDSVEKZHIQNCA3737RMXUAOJ2S3LAD2DIHVLTA, seg_pos={0 0}, upload=55998]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: write tcp 139.59.144.22:48200->77.90.31.18:4807: use of closed network connection; write tcp 139.59.144.22:48200->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48200->77.90.31.18:4807: use of closed network connection Jul 26 21:59:12.841872124 (58h23m44.032170692s) [seg_pos={0 0}, upload=55998]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:59:12.841975383 (58h23m44.032273954s) [upload=55998]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:59:12.841981090 (58h23m44.032279669s) [upload=55998]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:59:13.007332906 (58h23m44.197631473s) [upload=55998]: Done uploading segments... Jul 26 21:59:13.007353428 (58h23m44.197651993s) [upload=55998]: Upload finished. err: <nil> Duration: 30.004212352s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 9.042913856s FAIL (2025-07-26 21:59:15 UTC) orbiter warning: context deadline exceeded Duration: 9.042972887s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XX.X.............] duration: 22.210040019s FAIL (2025-07-26 21:59:13 UTC) orbiter warning: uplink: context deadline exceeded Duration: 22.210077322s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.01760768s FAIL (2025-07-26 21:58:51 UTC) orbiter warning: context deadline exceeded Duration: 6.017633276s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................X.X..X..X] duration: 30.006139028s FAIL (2025-07-26 21:58:26 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 52->77.90.31.13:4626: use of closed network connection; write tcp 139.59.144.22:60552->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:60552->77.90.31.13:4626: use of closed network connection Jul 26 21:57:26.988720987 (58h21m58.179019568s) [seg_pos={0 0}, upload=55978]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:57:26.988810471 (58h21m58.179109042s) [upload=55978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:57:26.988816308 (58h21m58.179114887s) [upload=55978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:57:27.140602780 (58h21m58.330901368s) [upload=55978]: Done uploading segments... Jul 26 21:57:27.140629457 (58h21m58.330928044s) [upload=55978]: Upload finished. err: <nil> Jul 26 21:57:56.869454509 (58h22m28.059753095s) [upload=55984]: Starting upload Jul 26 21:57:56.869615458 (58h22m28.059914044s) [upload=55984]: Done starting upload Jul 26 21:57:56.869745635 (58h22m28.060044206s) [upload=55984]: Uploading segments... Jul 26 21:57:56.870054280 (58h22m28.060352869s) [upload=55984]: Got next segment. Inline: false Jul 26 21:57:56.870132762 (58h22m28.060431346s) [upload=55984]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:57:56.870142204 (58h22m28.060440887s) [upload=55984]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:57:56.978158631 (58h22m28.16845722s) [seg_pos={0 0}, upload=55984]: Begin upload segment... Jul 26 21:57:56.978248062 (58h22m28.168546662s) [seg_pos={0 0}, upload=55984]: Done begin upload segment. Jul 26 21:57:56.978315555 (58h22m28.168614157s) [noise=true , address=45.91.116.9:4262 , piece_id=TB4NWLTH5DQ7QWXPPUOLCWDD5SM5JKKQXXFHK2AHYVJWBSNZ7BKA, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.978392236 (58h22m28.168690825s) [noise=true , address=104.36.82.4:4801 , piece_id=NEOYOJTNONLC23Y4TUPET43UOW7QFOI4L5OS4SWPDRSHEQUZL64Q, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.979834213 (58h22m28.170132812s) [noise=true , address=104.36.82.8:4970 , piece_id=YTLUVD6HKUF64TGN57L4J2AMXADSPQ3XMHC3INJFDSAVTBTT6D6A, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.980304595 (58h22m28.170603186s) [noise=true , address=45.91.116.4:4937 , piece_id=B2F5ZBAMGS3SQC6WIBQENW4EWWFXJJ5OKP4II32HGVZWDCMQ3QKA, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.980638443 (58h22m28.170937036s) [noise=true , address=45.91.116.5:4299 , piece_id=HCCVB5DOFCCJS4VPZDACQD777T4N7Z2MW4EYFOSXZF37RTIIFN2A, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.981255500 (58h22m28.171554089s) [noise=true , address=45.91.116.6:4975 , piece_id=DIH53PNPHLELHVI64JMNCGWSQVEJTQTZEHI3BXQIUOVDFLELCMZQ, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.982676235 (58h22m28.172974826s) [noise=true , address=104.36.82.8:4969 , piece_id=4S6Q6VN4RZFSD337BU6EGXPLMBV2TOPPBOMJTNWSWG73EYRNQSVQ, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.984421339 (58h22m28.174719928s) [noise=true , address=45.91.116.12:4226 , piece_id=FSPDAFKJUVTLTMD5AJS7WNT4X3GRQR3ZAJYJDGHYXDZ3WNHPISSQ, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.985712589 (58h22m28.176011167s) [noise=true , address=45.91.116.7:4011 , piece_id=YDLJDDINR3IMGAENCMEBZ4SSNIASQ67U23MJI2VN4Z7DPYCUNQXA, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.986391650 (58h22m28.17669023s) [noise=true , address=45.91.116.7:4009 , piece_id=KXUQAUX3XBDE7QFAVW247KSSSM6KAH43VM6J7EAQSPY3C6Z4K6EQ, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.986564969 (58h22m28.17686358s) [noise=true , address=45.91.116.13:4155 , piece_id=LMIXKMNEIJ7RDZFURUURSEKGE4YB7Q72KGAVMCH7NGLDAM5GCLZA, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.987023982 (58h22m28.177322565s) [noise=true , address=104.36.82.12:4150 , piece_id=JFZNY3J74PDL7NNU55ZPCT2RZEHVLDKGBNAKOTLBPJSTAJFZBPZQ, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.987368425 (58h22m28.177667017s) [noise=true , address=45.91.116.8:4047 , piece_id=TJXACWX57HETBMMD3I5BQ7ZENI6UFHK4ZFMZQ2HMIXNYBVWXR26A, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.988462270 (58h22m28.178760858s) [noise=true , address=77.90.31.16:4735 , piece_id=42R4BTM4OIZ36ONN4FDGELDV6U7RSVJ7UBYZV54TLWUV5BF5M6EA, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.988998831 (58h22m28.179297414s) [noise=true , address=45.91.116.8:4048 , piece_id=6SAYJEAYJX7ZCUKC4ARRST62OAGWFKP6JYBJYX3V5UUUJB6YUPWA, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:56.989437691 (58h22m28.179736279s) [noise=true , address=104.36.82.2:4732 , piece_id=FEAVG5CFSSXQM3UDV25QTC5LBAGXFJML5INI77VZCNQXEYLZJUNQ, seg_pos={0 0}, upload=55984]: Uploading piece... Jul 26 21:57:57.609763441 (58h22m28.800062031s) [upload=55984]: Next returned nil segment Jul 26 21:57:57.609791006 (58h22m28.8000896s) [upload=55984]: Waiting for error group managing segments... Jul 26 21:57:57.742953015 (58h22m28.933251598s) [noise=true , address=45.91.116.7:4011 , piece_id=YDLJDDINR3IMGAENCMEBZ4SSNIASQ67U23MJI2VN4Z7DPYCUNQXA, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.743135244 (58h22m28.933433824s) [noise=true , address=45.91.116.12:4226 , piece_id=FSPDAFKJUVTLTMD5AJS7WNT4X3GRQR3ZAJYJDGHYXDZ3WNHPISSQ, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.744676647 (58h22m28.934975229s) [noise=true , address=45.91.116.9:4262 , piece_id=TB4NWLTH5DQ7QWXPPUOLCWDD5SM5JKKQXXFHK2AHYVJWBSNZ7BKA, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.746308765 (58h22m28.936607338s) [noise=true , address=45.91.116.5:4299 , piece_id=HCCVB5DOFCCJS4VPZDACQD777T4N7Z2MW4EYFOSXZF37RTIIFN2A, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.749585327 (58h22m28.939883926s) [noise=true , address=45.91.116.6:4975 , piece_id=DIH53PNPHLELHVI64JMNCGWSQVEJTQTZEHI3BXQIUOVDFLELCMZQ, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.752889112 (58h22m28.943187684s) [noise=true , address=45.91.116.8:4047 , piece_id=TJXACWX57HETBMMD3I5BQ7ZENI6UFHK4ZFMZQ2HMIXNYBVWXR26A, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.758464642 (58h22m28.948763221s) [noise=true , address=45.91.116.4:4937 , piece_id=B2F5ZBAMGS3SQC6WIBQENW4EWWFXJJ5OKP4II32HGVZWDCMQ3QKA, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.765449251 (58h22m28.955747823s) [noise=true , address=45.91.116.8:4048 , piece_id=6SAYJEAYJX7ZCUKC4ARRST62OAGWFKP6JYBJYX3V5UUUJB6YUPWA, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.771929490 (58h22m28.96222806s) [noise=true , address=45.91.116.13:4155 , piece_id=LMIXKMNEIJ7RDZFURUURSEKGE4YB7Q72KGAVMCH7NGLDAM5GCLZA, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:57:57.774068298 (58h22m28.964366871s) [noise=true , address=45.91.116.7:4009 , piece_id=KXUQAUX3XBDE7QFAVW247KSSSM6KAH43VM6J7EAQSPY3C6Z4K6EQ, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:58:15.759688842 (58h22m46.949987417s) [noise=true , address=77.90.31.16:4735 , piece_id=42R4BTM4OIZ36ONN4FDGELDV6U7RSVJ7UBYZV54TLWUV5BF5M6EA, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:58:21.574674460 (58h22m52.76497303s) [noise=true , address=104.36.82.12:4150 , piece_id=JFZNY3J74PDL7NNU55ZPCT2RZEHVLDKGBNAKOTLBPJSTAJFZBPZQ, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:58:25.196906058 (58h22m56.387204631s) [noise=true , address=104.36.82.8:4969 , piece_id=4S6Q6VN4RZFSD337BU6EGXPLMBV2TOPPBOMJTNWSWG73EYRNQSVQ, seg_pos={0 0}, upload=55984]: Done uploading piece. err: <nil> Jul 26 21:58:25.196937337 (58h22m56.387235916s) [seg_pos={0 0}, upload=55984]: Segment reached optimal threshold of 13 pieces. Jul 26 21:58:25.197088517 (58h22m56.387387089s) [noise=true , address=104.36.82.2:4732 , piece_id=FEAVG5CFSSXQM3UDV25QTC5LBAGXFJML5INI77VZCNQXEYLZJUNQ, seg_pos={0 0}, upload=55984]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Jul 26 21:58:25.197208239 (58h22m56.38750681s) [noise=true , address=104.36.82.8:4970 , piece_id=YTLUVD6HKUF64TGN57L4J2AMXADSPQ3XMHC3INJFDSAVTBTT6D6A, seg_pos={0 0}, upload=55984]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe): protocol: expected piece hash; context canceled; EOF Jul 26 21:58:25.197302836 (58h22m56.387601429s) [noise=true , address=104.36.82.4:4801 , piece_id=NEOYOJTNONLC23Y4TUPET43UOW7QFOI4L5OS4SWPDRSHEQUZL64Q, seg_pos={0 0}, upload=55984]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 139.59.144.22:56618->104.36.82.4:4801: use of closed network connection; write tcp 139.59.144.22:56618->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56618->104.36.82.4:4801: use of closed network connection Jul 26 21:58:25.197342845 (58h22m56.387641425s) [seg_pos={0 0}, upload=55984]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:58:25.197437160 (58h22m56.38773575s) [upload=55984]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:58:25.197443230 (58h22m56.387741816s) [upload=55984]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:58:25.352579068 (58h22m56.542877652s) [upload=55984]: Done uploading segments... Jul 26 21:58:25.352624830 (58h22m56.542923402s) [upload=55984]: Upload finished. err: <nil> Duration: 30.006171443s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)