Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.2951461s FAIL (2025-06-24 15:20:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.29517668s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 32.324960937s FAIL (2025-06-24 15:20:54 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 32.324997344s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.477921628s FAIL (2025-06-24 15:20:18 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.47797425s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 918.12676ms FAIL (2025-06-24 15:20:14 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 918.158609ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.323788088s FAIL (2025-06-24 15:19:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.323822079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.410760306s FAIL (2025-06-24 15:19:57 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.410786149s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 867.517132ms FAIL (2025-06-24 15:19:46 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 867.546404ms edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.014975745s FAIL (2025-06-24 15:19:40 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 1.015024364s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 3.184588394s FAIL (2025-06-24 15:19:26 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.184619493s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 938.021804ms FAIL (2025-06-24 15:19:20 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 938.055786ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.504454047s FAIL (2025-06-24 15:18:54 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.504485567s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.278500124s FAIL (2025-06-24 15:18:48 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.278528621s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 3.021901419s FAIL (2025-06-24 15:18:32 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.021940719s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 876.31801ms FAIL (2025-06-24 15:18:32 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 876.349418ms edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 886.273089ms FAIL (2025-06-24 15:18:25 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 886.304177ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.634273629s FAIL (2025-06-24 15:18:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.634313604s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.309062583s FAIL (2025-06-24 15:18:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.309097199s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.274689989s FAIL (2025-06-24 15:17:49 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.274718735s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 917.304717ms FAIL (2025-06-24 15:17:38 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 917.336153ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 3.67836304s FAIL (2025-06-24 15:17:26 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.678385216s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 2.662729487s FAIL (2025-06-24 15:17:21 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.662761403s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 876.084946ms FAIL (2025-06-24 15:17:17 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 876.111243ms edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.371534777s FAIL (2025-06-24 15:16:43 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.371586714s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 927.203883ms FAIL (2025-06-24 15:16:39 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 927.229574ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 3.469056514s FAIL (2025-06-24 15:16:34 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.469087258s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.837542178s FAIL (2025-06-24 15:16:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.837589402s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 902.888549ms FAIL (2025-06-24 15:16:15 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 902.916698ms edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.349976655s FAIL (2025-06-24 15:16:11 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.350006565s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 3.248617681s FAIL (2025-06-24 15:15:53 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.248667726s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.069105636s FAIL (2025-06-24 15:15:39 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 1.069137305s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 3.506258152s FAIL (2025-06-24 15:15:36 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.506283106s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.265651093s FAIL (2025-06-24 15:15:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.265675823s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.382805796s FAIL (2025-06-24 15:15:26 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.382836221s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.377386412s FAIL (2025-06-24 15:15:06 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.377476093s core uplink us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 642.939834ms FAIL (2025-06-24 15:15:04 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 643.030201ms edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 3.647119323s FAIL (2025-06-24 15:14:50 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.647150577s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 894.811004ms FAIL (2025-06-24 15:14:44 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 894.844102ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.388150276s FAIL (2025-06-24 15:14:28 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.388178311s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 882.664573ms FAIL (2025-06-24 15:14:18 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 882.691538ms edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.339258263s FAIL (2025-06-24 15:14:07 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.339299759s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.246140426s FAIL (2025-06-24 15:14:00 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.246172035s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 3.144288245s FAIL (2025-06-24 15:13:57 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.144312733s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 864.219105ms FAIL (2025-06-24 15:13:56 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 864.277405ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.224887469s FAIL (2025-06-24 15:13:24 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.224922229s core uplink us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1.408362467s FAIL (2025-06-24 15:13:10 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 1.408389557s edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.544038861s FAIL (2025-06-24 15:13:01 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== 38.710079463 (29h51m38.593493084s) [seg_pos={0 0}, upload=23996]: Done begin upload segment. Jun 24 15:02:38.710129041 (29h51m38.59354267s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710138687 (29h51m38.593552328s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710690489 (29h51m38.594104106s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.710713779 (29h51m38.594127417s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711106997 (29h51m38.594520623s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.711531376 (29h51m38.594945005s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712074545 (29h51m38.59548818s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712237061 (29h51m38.595650701s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712469485 (29h51m38.595883111s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712638648 (29h51m38.596052285s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.712933855 (29h51m38.596347485s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713378835 (29h51m38.596792472s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713813822 (29h51m38.597227453s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.713901007 (29h51m38.597314653s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714349767 (29h51m38.597763401s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.714694647 (29h51m38.59810828s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715041886 (29h51m38.598455519s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:38.715462120 (29h51m38.59887574s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Uploading piece... Jun 24 15:02:39.251183574 (29h51m39.134597192s) [upload=23996]: Next returned nil segment Jun 24 15:02:39.251207849 (29h51m39.134621467s) [upload=23996]: Waiting for error group managing segments... Jun 24 15:02:39.412884907 (29h51m39.296298523s) [noise=true , address=104.36.82.6:5906 , piece_id=YRR3B6FBQM4YWFB4MHDYE3N7RUWED2MDX7243GOIDBK2CVZ6J4VQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.417725225 (29h51m39.301138871s) [noise=true , address=104.36.82.21:6489 , piece_id=ZEGGOTZJDEGFSMJZBVHMAHYUI34KRMXUYTUF2W2M3RAVD72RWT4A, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.419058833 (29h51m39.302472446s) [noise=true , address=45.91.116.14:4198 , piece_id=G2W6P2PKFQRNTHLPS3DBHQX4RCJBWHHHWKA27H33V5WOTOD7QSCQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.428264821 (29h51m39.311678434s) [noise=true , address=104.36.82.53:4128 , piece_id=EWY4IUE6MBFEQZ3OAP3KGDVL23S33RZZBGWB66NLDZWJSUQNBKZQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.430916704 (29h51m39.314330317s) [noise=true , address=45.91.116.10:4100 , piece_id=4JCWPFVKQ4ETQIURSDUERMDBLNL2CLC7VCY22NEZEK2A4Y33WJOQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.441395180 (29h51m39.324808793s) [noise=true , address=104.36.82.42:6831 , piece_id=J2LMKWWXSZIQQVWEUM6LOXV25Q4BEVAR6ZN4VCBTFF3GI2SM7UDQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.443411058 (29h51m39.326824675s) [noise=true , address=104.36.82.52:4112 , piece_id=WMB63X5QWSZS4XBCLAIEHFGHOSN33PPTQDWBGJSYOHTDTWLAGJOA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.453557685 (29h51m39.336971305s) [noise=true , address=104.36.82.55:4221 , piece_id=7EXOD4W5KO3HUB7CEG7UNQO2JZ6QQBOKUJIDVGMTEX3OQ5POTPHA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.461322189 (29h51m39.344735804s) [noise=true , address=104.36.82.8:4003 , piece_id=5UI755XLYORU7BA6DBG45SWXWZUP44WZU5W7KNBJOVLMHAMK4IEA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473469757 (29h51m39.356883373s) [noise=true , address=79.127.219.38:4017 , piece_id=V3ANYU4D6FIQZ74TIBKBISJLOAC7MZTZ4D7WBDN64Z4CLFGYPDVA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.473537251 (29h51m39.356950869s) [noise=true , address=104.36.82.10:4092 , piece_id=VJG4BJEDX4BSBI3U2RTBSAHZCITV52LCRQZ4T6URXBKRQUYJZX4Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.486281273 (29h51m39.369694894s) [noise=true , address=79.127.219.38:4015 , piece_id=NMVYGH5KKKRKD5DPT7S2QL2DAUXIY7IO4BCBISJKOJJVW7QIYUMQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.571776596 (29h51m39.455190214s) [noise=true , address=79.127.220.101:6048 , piece_id=SATKRPLQAM4DRYM65WQDFMCDET3Q2USQWOBQ7XNXFFTXWBSBHFWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653833016 (29h51m39.537246631s) [noise=true , address=79.127.220.97:4372 , piece_id=FHNSBGXZ7VGYLC2RHO7SPXS5AKPF3MMEXASE5W75UHKFLHWROXAQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: <nil> Jun 24 15:02:39.653863972 (29h51m39.537277598s) [seg_pos={0 0}, upload=23996]: Segment reached optimal threshold of 14 pieces. Jun 24 15:02:39.654154687 (29h51m39.537568301s) [noise=true , address=104.36.82.5:4858 , piece_id=KBUDGWIRVHFPQTLPPHHYPMJVSVAE7CMR2CGZ37YUGRUK42I2BT7Q, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PasA9NPExHCQgMSK1E1kz2HHkpXQmWq7LmrwY6sRmf22avri): protocol: expected piece hash; context canceled; EOF Jun 24 15:02:39.654204520 (29h51m39.537618156s) [noise=true , address=64.49.13.140:4513 , piece_id=4ZABXGCNM3PM64262VCM5IOB4BEWYPY4PARANALWVCOEX5WQKDWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY): protocol: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:48192->64.49.13.140:4513: use of closed network connection Jun 24 15:02:39.654312246 (29h51m39.537725915s) [noise=true , address=64.49.13.169:4705 , piece_id=PIOJN7FOHRP6HVGXMYQRQDQJURVRBMUMTW7H4C3CB6W24ZP76WPQ, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124uqzVwsF5TM4CwUhkzfJvgxd71jZXgXuFNwDLRW1vrSXbwLsr): protocol: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40656->64.49.13.169:4705: use of closed network connection Jun 24 15:02:39.654354065 (29h51m39.537767695s) [noise=true , address=79.127.205.235:5682 , piece_id=OMZRVAP7JEPKG3ISG25OLLL3IBE53FKZ5DNTOM6HNXMBSHJ5JKWA, seg_pos={0 0}, upload=23996]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZgWv9RA5EpAHfxzBWLpTj8un93LPU8syqxM9gBQQomMP9MRwG): protocol: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49606->79.127.205.235:5682: use of closed network connection Jun 24 15:02:39.654401800 (29h51m39.537815437s) [seg_pos={0 0}, upload=23996]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jun 24 15:02:39.654504756 (29h51m39.537918371s) [upload=23996]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 15:02:39.654510115 (29h51m39.537923741s) [upload=23996]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 15:02:39.810746072 (29h51m39.694159689s) [upload=23996]: Done uploading segments... Jun 24 15:02:39.810767325 (29h51m39.694180941s) [upload=23996]: Upload finished. err: <nil> Duration: 2.544074755s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 3.188171363s FAIL (2025-06-24 15:12:51 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "\n" Duration: 3.188195762s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.182378525s FAIL (2025-06-24 15:12:46 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 1.182408943s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.834711892s FAIL (2025-06-24 15:12:19 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments... Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil> Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments... Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment... Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment. Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece... Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments... Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil> Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces. Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments... Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil> Duration: 2.834753272s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.27270626s FAIL (2025-06-24 15:12:14 UTC) orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil> Duration: 1.273033005s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)