Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 31 out of 60 is failed (threshold=3) [...XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XXXXX.XX.] duration: 48.497042007s FAIL (2025-07-09 01:46:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 48.497064627s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X.X.XXXXX.......XX....................X..X..X.............X.] duration: 16.095561978s FAIL (2025-07-09 01:46:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.095587976s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X.X.XXXXX.......XX....................X..X..X.............X] duration: 30.000673125s FAIL (2025-07-09 01:45:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000694296s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 31 out of 60 is failed (threshold=3) [....XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XXXXX.XX] duration: 1m0.001024748s FAIL (2025-07-09 01:45:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001044192s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X...................................X.....] duration: 2.390164666s FAIL (2025-07-09 01:45:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== UARKKKYWHS6OJARTQ, seg_pos={0 0}, upload=301070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 9 01:40:04.447461354 (376h29m4.330874974s) [seg_pos={0 0}, upload=301070]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 01:40:04.447566480 (376h29m4.3309801s) [upload=301070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:40:04.447573285 (376h29m4.330986904s) [upload=301070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:40:04.593277678 (376h29m4.476691297s) [upload=301070]: Done uploading segments... Jul 9 01:40:04.593300848 (376h29m4.476714464s) [upload=301070]: Upload finished. err: <nil> Jul 9 01:41:14.682414521 (376h30m14.565828152s) [upload=301082]: Starting upload Jul 9 01:41:14.682503694 (376h30m14.565917327s) [upload=301082]: Done starting upload Jul 9 01:41:14.682895930 (376h30m14.566309543s) [upload=301082]: Uploading segments... Jul 9 01:41:14.682966356 (376h30m14.566379999s) [upload=301082]: Got next segment. Inline: false Jul 9 01:41:14.683011792 (376h30m14.566425428s) [upload=301082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:41:14.683017185 (376h30m14.566430818s) [upload=301082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:41:14.790212148 (376h30m14.673625784s) [seg_pos={0 0}, upload=301082]: Begin upload segment... Jul 9 01:41:14.790319542 (376h30m14.673733178s) [seg_pos={0 0}, upload=301082]: Done begin upload segment. Jul 9 01:41:14.790497270 (376h30m14.673910907s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.790540123 (376h30m14.67395376s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.791164639 (376h30m14.674578277s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.792366139 (376h30m14.675779767s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.793650193 (376h30m14.677064072s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.794895159 (376h30m14.678308796s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.795507503 (376h30m14.678921142s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.796576755 (376h30m14.679990392s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.798463453 (376h30m14.681877094s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.799335495 (376h30m14.682749139s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.800229900 (376h30m14.683643538s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.806816135 (376h30m14.690229772s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.808677646 (376h30m14.692091277s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.810758199 (376h30m14.694171828s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.812672044 (376h30m14.696085673s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.814773304 (376h30m14.698186935s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:15.406098847 (376h30m15.289512467s) [upload=301082]: Next returned nil segment Jul 9 01:41:15.406117780 (376h30m15.289531408s) [upload=301082]: Waiting for error group managing segments... Jul 9 01:41:15.538146641 (376h30m15.42156026s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.539592447 (376h30m15.423006066s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540032848 (376h30m15.423446465s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540704801 (376h30m15.424118441s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.557783745 (376h30m15.441197365s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.558361735 (376h30m15.441775352s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.565538086 (376h30m15.448951702s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.571767616 (376h30m15.455181304s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.573630878 (376h30m15.457044504s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.580306535 (376h30m15.463720154s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.586995475 (376h30m15.470409092s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:37.968158647 (376h30m37.851572274s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:44.413211932 (376h30m44.296625561s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413322687 (376h30m44.296736315s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413452023 (376h30m44.296865635s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:54328->104.36.82.4:4801: use of closed network connection; context deadline exceeded Jul 9 01:41:44.413471848 (376h30m44.296885488s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection Jul 9 01:41:44.413539946 (376h30m44.29695356s) [seg_pos={0 0}, upload=301082]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:41:44.577747284 (376h30m44.461160908s) [upload=301082]: Done uploading segments... Jul 9 01:41:44.577784030 (376h30m44.461197666s) [upload=301082]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.390287435s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..X.X.XXXXX.......XX....................X..X..X.............] duration: 2.610086208s FAIL (2025-07-09 01:44:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.610116601s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 30 out of 60 is failed (threshold=3) [.....XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XXXXX.X] duration: 1m0.000669301s FAIL (2025-07-09 01:44:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000694527s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X...................................X....] duration: 2.458584226s FAIL (2025-07-09 01:44:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== UARKKKYWHS6OJARTQ, seg_pos={0 0}, upload=301070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 9 01:40:04.447461354 (376h29m4.330874974s) [seg_pos={0 0}, upload=301070]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 01:40:04.447566480 (376h29m4.3309801s) [upload=301070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:40:04.447573285 (376h29m4.330986904s) [upload=301070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:40:04.593277678 (376h29m4.476691297s) [upload=301070]: Done uploading segments... Jul 9 01:40:04.593300848 (376h29m4.476714464s) [upload=301070]: Upload finished. err: <nil> Jul 9 01:41:14.682414521 (376h30m14.565828152s) [upload=301082]: Starting upload Jul 9 01:41:14.682503694 (376h30m14.565917327s) [upload=301082]: Done starting upload Jul 9 01:41:14.682895930 (376h30m14.566309543s) [upload=301082]: Uploading segments... Jul 9 01:41:14.682966356 (376h30m14.566379999s) [upload=301082]: Got next segment. Inline: false Jul 9 01:41:14.683011792 (376h30m14.566425428s) [upload=301082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:41:14.683017185 (376h30m14.566430818s) [upload=301082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:41:14.790212148 (376h30m14.673625784s) [seg_pos={0 0}, upload=301082]: Begin upload segment... Jul 9 01:41:14.790319542 (376h30m14.673733178s) [seg_pos={0 0}, upload=301082]: Done begin upload segment. Jul 9 01:41:14.790497270 (376h30m14.673910907s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.790540123 (376h30m14.67395376s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.791164639 (376h30m14.674578277s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.792366139 (376h30m14.675779767s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.793650193 (376h30m14.677064072s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.794895159 (376h30m14.678308796s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.795507503 (376h30m14.678921142s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.796576755 (376h30m14.679990392s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.798463453 (376h30m14.681877094s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.799335495 (376h30m14.682749139s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.800229900 (376h30m14.683643538s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.806816135 (376h30m14.690229772s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.808677646 (376h30m14.692091277s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.810758199 (376h30m14.694171828s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.812672044 (376h30m14.696085673s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.814773304 (376h30m14.698186935s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:15.406098847 (376h30m15.289512467s) [upload=301082]: Next returned nil segment Jul 9 01:41:15.406117780 (376h30m15.289531408s) [upload=301082]: Waiting for error group managing segments... Jul 9 01:41:15.538146641 (376h30m15.42156026s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.539592447 (376h30m15.423006066s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540032848 (376h30m15.423446465s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540704801 (376h30m15.424118441s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.557783745 (376h30m15.441197365s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.558361735 (376h30m15.441775352s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.565538086 (376h30m15.448951702s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.571767616 (376h30m15.455181304s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.573630878 (376h30m15.457044504s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.580306535 (376h30m15.463720154s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.586995475 (376h30m15.470409092s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:37.968158647 (376h30m37.851572274s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:44.413211932 (376h30m44.296625561s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413322687 (376h30m44.296736315s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413452023 (376h30m44.296865635s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:54328->104.36.82.4:4801: use of closed network connection; context deadline exceeded Jul 9 01:41:44.413471848 (376h30m44.296885488s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection Jul 9 01:41:44.413539946 (376h30m44.29695356s) [seg_pos={0 0}, upload=301082]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:41:44.577747284 (376h30m44.461160908s) [upload=301082]: Done uploading segments... Jul 9 01:41:44.577784030 (376h30m44.461197666s) [upload=301082]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.458619064s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...X.X.XXXXX.......XX....................X..X..X............] duration: 2.707656137s FAIL (2025-07-09 01:43:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.707683164s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X...................................X...] duration: 2.606509118s FAIL (2025-07-09 01:43:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== UARKKKYWHS6OJARTQ, seg_pos={0 0}, upload=301070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 9 01:40:04.447461354 (376h29m4.330874974s) [seg_pos={0 0}, upload=301070]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 01:40:04.447566480 (376h29m4.3309801s) [upload=301070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:40:04.447573285 (376h29m4.330986904s) [upload=301070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:40:04.593277678 (376h29m4.476691297s) [upload=301070]: Done uploading segments... Jul 9 01:40:04.593300848 (376h29m4.476714464s) [upload=301070]: Upload finished. err: <nil> Jul 9 01:41:14.682414521 (376h30m14.565828152s) [upload=301082]: Starting upload Jul 9 01:41:14.682503694 (376h30m14.565917327s) [upload=301082]: Done starting upload Jul 9 01:41:14.682895930 (376h30m14.566309543s) [upload=301082]: Uploading segments... Jul 9 01:41:14.682966356 (376h30m14.566379999s) [upload=301082]: Got next segment. Inline: false Jul 9 01:41:14.683011792 (376h30m14.566425428s) [upload=301082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:41:14.683017185 (376h30m14.566430818s) [upload=301082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:41:14.790212148 (376h30m14.673625784s) [seg_pos={0 0}, upload=301082]: Begin upload segment... Jul 9 01:41:14.790319542 (376h30m14.673733178s) [seg_pos={0 0}, upload=301082]: Done begin upload segment. Jul 9 01:41:14.790497270 (376h30m14.673910907s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.790540123 (376h30m14.67395376s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.791164639 (376h30m14.674578277s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.792366139 (376h30m14.675779767s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.793650193 (376h30m14.677064072s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.794895159 (376h30m14.678308796s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.795507503 (376h30m14.678921142s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.796576755 (376h30m14.679990392s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.798463453 (376h30m14.681877094s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.799335495 (376h30m14.682749139s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.800229900 (376h30m14.683643538s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.806816135 (376h30m14.690229772s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.808677646 (376h30m14.692091277s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.810758199 (376h30m14.694171828s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.812672044 (376h30m14.696085673s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.814773304 (376h30m14.698186935s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:15.406098847 (376h30m15.289512467s) [upload=301082]: Next returned nil segment Jul 9 01:41:15.406117780 (376h30m15.289531408s) [upload=301082]: Waiting for error group managing segments... Jul 9 01:41:15.538146641 (376h30m15.42156026s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.539592447 (376h30m15.423006066s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540032848 (376h30m15.423446465s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540704801 (376h30m15.424118441s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.557783745 (376h30m15.441197365s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.558361735 (376h30m15.441775352s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.565538086 (376h30m15.448951702s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.571767616 (376h30m15.455181304s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.573630878 (376h30m15.457044504s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.580306535 (376h30m15.463720154s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.586995475 (376h30m15.470409092s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:37.968158647 (376h30m37.851572274s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:44.413211932 (376h30m44.296625561s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413322687 (376h30m44.296736315s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413452023 (376h30m44.296865635s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:54328->104.36.82.4:4801: use of closed network connection; context deadline exceeded Jul 9 01:41:44.413471848 (376h30m44.296885488s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection Jul 9 01:41:44.413539946 (376h30m44.29695356s) [seg_pos={0 0}, upload=301082]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:41:44.577747284 (376h30m44.461160908s) [upload=301082]: Done uploading segments... Jul 9 01:41:44.577784030 (376h30m44.461197666s) [upload=301082]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.606540922s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 29 out of 60 is failed (threshold=3) [......XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XXXXX.] duration: 36.599878864s FAIL (2025-07-09 01:43:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 36.599892002s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....X.X.XXXXX.......XX....................X..X..X...........] duration: 18.697359786s FAIL (2025-07-09 01:43:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.69739097s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 29 out of 60 is failed (threshold=3) [.......XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XXXXX] duration: 1m0.001053379s FAIL (2025-07-09 01:42:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001070159s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X...................................X..] duration: 2.284854028s FAIL (2025-07-09 01:42:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== UARKKKYWHS6OJARTQ, seg_pos={0 0}, upload=301070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 9 01:40:04.447461354 (376h29m4.330874974s) [seg_pos={0 0}, upload=301070]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 01:40:04.447566480 (376h29m4.3309801s) [upload=301070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:40:04.447573285 (376h29m4.330986904s) [upload=301070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:40:04.593277678 (376h29m4.476691297s) [upload=301070]: Done uploading segments... Jul 9 01:40:04.593300848 (376h29m4.476714464s) [upload=301070]: Upload finished. err: <nil> Jul 9 01:41:14.682414521 (376h30m14.565828152s) [upload=301082]: Starting upload Jul 9 01:41:14.682503694 (376h30m14.565917327s) [upload=301082]: Done starting upload Jul 9 01:41:14.682895930 (376h30m14.566309543s) [upload=301082]: Uploading segments... Jul 9 01:41:14.682966356 (376h30m14.566379999s) [upload=301082]: Got next segment. Inline: false Jul 9 01:41:14.683011792 (376h30m14.566425428s) [upload=301082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:41:14.683017185 (376h30m14.566430818s) [upload=301082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:41:14.790212148 (376h30m14.673625784s) [seg_pos={0 0}, upload=301082]: Begin upload segment... Jul 9 01:41:14.790319542 (376h30m14.673733178s) [seg_pos={0 0}, upload=301082]: Done begin upload segment. Jul 9 01:41:14.790497270 (376h30m14.673910907s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.790540123 (376h30m14.67395376s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.791164639 (376h30m14.674578277s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.792366139 (376h30m14.675779767s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.793650193 (376h30m14.677064072s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.794895159 (376h30m14.678308796s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.795507503 (376h30m14.678921142s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.796576755 (376h30m14.679990392s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.798463453 (376h30m14.681877094s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.799335495 (376h30m14.682749139s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.800229900 (376h30m14.683643538s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.806816135 (376h30m14.690229772s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.808677646 (376h30m14.692091277s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.810758199 (376h30m14.694171828s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.812672044 (376h30m14.696085673s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.814773304 (376h30m14.698186935s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:15.406098847 (376h30m15.289512467s) [upload=301082]: Next returned nil segment Jul 9 01:41:15.406117780 (376h30m15.289531408s) [upload=301082]: Waiting for error group managing segments... Jul 9 01:41:15.538146641 (376h30m15.42156026s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.539592447 (376h30m15.423006066s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540032848 (376h30m15.423446465s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540704801 (376h30m15.424118441s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.557783745 (376h30m15.441197365s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.558361735 (376h30m15.441775352s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.565538086 (376h30m15.448951702s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.571767616 (376h30m15.455181304s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.573630878 (376h30m15.457044504s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.580306535 (376h30m15.463720154s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.586995475 (376h30m15.470409092s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:37.968158647 (376h30m37.851572274s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:44.413211932 (376h30m44.296625561s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413322687 (376h30m44.296736315s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413452023 (376h30m44.296865635s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:54328->104.36.82.4:4801: use of closed network connection; context deadline exceeded Jul 9 01:41:44.413471848 (376h30m44.296885488s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection Jul 9 01:41:44.413539946 (376h30m44.29695356s) [seg_pos={0 0}, upload=301082]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:41:44.577747284 (376h30m44.461160908s) [upload=301082]: Done uploading segments... Jul 9 01:41:44.577784030 (376h30m44.461197666s) [upload=301082]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.284881725s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X...................................X.] duration: 2.347920128s FAIL (2025-07-09 01:41:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== UARKKKYWHS6OJARTQ, seg_pos={0 0}, upload=301070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 9 01:40:04.447461354 (376h29m4.330874974s) [seg_pos={0 0}, upload=301070]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 01:40:04.447566480 (376h29m4.3309801s) [upload=301070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:40:04.447573285 (376h29m4.330986904s) [upload=301070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:40:04.593277678 (376h29m4.476691297s) [upload=301070]: Done uploading segments... Jul 9 01:40:04.593300848 (376h29m4.476714464s) [upload=301070]: Upload finished. err: <nil> Jul 9 01:41:14.682414521 (376h30m14.565828152s) [upload=301082]: Starting upload Jul 9 01:41:14.682503694 (376h30m14.565917327s) [upload=301082]: Done starting upload Jul 9 01:41:14.682895930 (376h30m14.566309543s) [upload=301082]: Uploading segments... Jul 9 01:41:14.682966356 (376h30m14.566379999s) [upload=301082]: Got next segment. Inline: false Jul 9 01:41:14.683011792 (376h30m14.566425428s) [upload=301082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:41:14.683017185 (376h30m14.566430818s) [upload=301082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:41:14.790212148 (376h30m14.673625784s) [seg_pos={0 0}, upload=301082]: Begin upload segment... Jul 9 01:41:14.790319542 (376h30m14.673733178s) [seg_pos={0 0}, upload=301082]: Done begin upload segment. Jul 9 01:41:14.790497270 (376h30m14.673910907s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.790540123 (376h30m14.67395376s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.791164639 (376h30m14.674578277s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.792366139 (376h30m14.675779767s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.793650193 (376h30m14.677064072s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.794895159 (376h30m14.678308796s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.795507503 (376h30m14.678921142s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.796576755 (376h30m14.679990392s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.798463453 (376h30m14.681877094s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.799335495 (376h30m14.682749139s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.800229900 (376h30m14.683643538s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.806816135 (376h30m14.690229772s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.808677646 (376h30m14.692091277s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.810758199 (376h30m14.694171828s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.812672044 (376h30m14.696085673s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.814773304 (376h30m14.698186935s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:15.406098847 (376h30m15.289512467s) [upload=301082]: Next returned nil segment Jul 9 01:41:15.406117780 (376h30m15.289531408s) [upload=301082]: Waiting for error group managing segments... Jul 9 01:41:15.538146641 (376h30m15.42156026s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.539592447 (376h30m15.423006066s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540032848 (376h30m15.423446465s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540704801 (376h30m15.424118441s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.557783745 (376h30m15.441197365s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.558361735 (376h30m15.441775352s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.565538086 (376h30m15.448951702s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.571767616 (376h30m15.455181304s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.573630878 (376h30m15.457044504s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.580306535 (376h30m15.463720154s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.586995475 (376h30m15.470409092s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:37.968158647 (376h30m37.851572274s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:44.413211932 (376h30m44.296625561s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413322687 (376h30m44.296736315s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413452023 (376h30m44.296865635s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:54328->104.36.82.4:4801: use of closed network connection; context deadline exceeded Jul 9 01:41:44.413471848 (376h30m44.296885488s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection Jul 9 01:41:44.413539946 (376h30m44.29695356s) [seg_pos={0 0}, upload=301082]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:41:44.577747284 (376h30m44.461160908s) [upload=301082]: Done uploading segments... Jul 9 01:41:44.577784030 (376h30m44.461197666s) [upload=301082]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.347951154s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.....X.X.XXXXX.......XX....................X..X..X..........] duration: 18.820953403s FAIL (2025-07-09 01:41:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 18.820982816s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X...................................X] duration: 30.166966396s FAIL (2025-07-09 01:41:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== UARKKKYWHS6OJARTQ, seg_pos={0 0}, upload=301070]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Jul 9 01:40:04.447461354 (376h29m4.330874974s) [seg_pos={0 0}, upload=301070]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 9 01:40:04.447566480 (376h29m4.3309801s) [upload=301070]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:40:04.447573285 (376h29m4.330986904s) [upload=301070]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:40:04.593277678 (376h29m4.476691297s) [upload=301070]: Done uploading segments... Jul 9 01:40:04.593300848 (376h29m4.476714464s) [upload=301070]: Upload finished. err: <nil> Jul 9 01:41:14.682414521 (376h30m14.565828152s) [upload=301082]: Starting upload Jul 9 01:41:14.682503694 (376h30m14.565917327s) [upload=301082]: Done starting upload Jul 9 01:41:14.682895930 (376h30m14.566309543s) [upload=301082]: Uploading segments... Jul 9 01:41:14.682966356 (376h30m14.566379999s) [upload=301082]: Got next segment. Inline: false Jul 9 01:41:14.683011792 (376h30m14.566425428s) [upload=301082]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:41:14.683017185 (376h30m14.566430818s) [upload=301082]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:41:14.790212148 (376h30m14.673625784s) [seg_pos={0 0}, upload=301082]: Begin upload segment... Jul 9 01:41:14.790319542 (376h30m14.673733178s) [seg_pos={0 0}, upload=301082]: Done begin upload segment. Jul 9 01:41:14.790497270 (376h30m14.673910907s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.790540123 (376h30m14.67395376s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.791164639 (376h30m14.674578277s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.792366139 (376h30m14.675779767s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.793650193 (376h30m14.677064072s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.794895159 (376h30m14.678308796s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.795507503 (376h30m14.678921142s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.796576755 (376h30m14.679990392s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.798463453 (376h30m14.681877094s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.799335495 (376h30m14.682749139s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.800229900 (376h30m14.683643538s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.806816135 (376h30m14.690229772s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.808677646 (376h30m14.692091277s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.810758199 (376h30m14.694171828s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.812672044 (376h30m14.696085673s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:14.814773304 (376h30m14.698186935s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Uploading piece... Jul 9 01:41:15.406098847 (376h30m15.289512467s) [upload=301082]: Next returned nil segment Jul 9 01:41:15.406117780 (376h30m15.289531408s) [upload=301082]: Waiting for error group managing segments... Jul 9 01:41:15.538146641 (376h30m15.42156026s) [noise=true , address=45.91.116.14:4190 , piece_id=FNP5PTK6DKU4TYBYAEF6FZCIKXMOM2PDB32UFF67GWVQ4LQJDRRQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.539592447 (376h30m15.423006066s) [noise=true , address=45.91.116.12:4226 , piece_id=I4TVQXPSTRUQUWN4IQLDFZ3SCRHBCI2ZTB5KKQF4H56RQUEWZXYQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540032848 (376h30m15.423446465s) [noise=true , address=45.91.116.9:4262 , piece_id=W4G36J5XZFKTAJNB4VBI55ECCSB3ZXHF2SPD3JG7XECUYCWED6HQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.540704801 (376h30m15.424118441s) [noise=true , address=45.91.116.10:4082 , piece_id=S7R3NZTPYD3SX66R3G62KRN6PJMAABDM2WLPUZ5ZICQWLYRJ3GTA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.557783745 (376h30m15.441197365s) [noise=true , address=45.91.116.13:4154 , piece_id=E7PRQBKNZ7VOL32UIH4HSXRZTBRJMUYYZ4DMQKTCS4OPNCBJ27IQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.558361735 (376h30m15.441775352s) [noise=true , address=45.91.116.5:4298 , piece_id=OA2LPGGPRITPZO4VT5REC42ONAEWXDYDMJAQ554HZXI2DJWEPAQQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.565538086 (376h30m15.448951702s) [noise=true , address=77.90.31.10:4518 , piece_id=FC23ZY2HNFKEMN5G3TLNYSJ6QTS57ELGGI5MSOS4OFKXWQJZH5OA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.571767616 (376h30m15.455181304s) [noise=true , address=45.91.116.6:4975 , piece_id=N3LDJ4G6XMWGUY2TEPDY4P5VJWOUMFOJEG3GMIQQONUTSM4UC6HA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.573630878 (376h30m15.457044504s) [noise=true , address=45.91.116.8:4045 , piece_id=CSZWJ3AJVDQHQZVPL3GPAI242XDUNIE52KWGT3KICAL3UA6MJBEA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.580306535 (376h30m15.463720154s) [noise=true , address=77.90.31.16:4733 , piece_id=XV73XHPUUVNC5LMBATS446ILA447SB7QIHBVWL2VBMIQJP5OHB4Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:15.586995475 (376h30m15.470409092s) [noise=true , address=77.90.31.19:4841 , piece_id=DNRBFG2SFYG3DYISXPXE4TJZ7PLKUWUJL2Z2BKR24PVCYWWJDY5Q, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:37.968158647 (376h30m37.851572274s) [noise=true , address=104.36.82.2:4730 , piece_id=H4ER6OZ6FG3H7LWCNTJM2Q5D2FLULXCVWHSAM6KABCAZ2BYEBQCQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: <nil> Jul 9 01:41:44.413211932 (376h30m44.296625561s) [noise=true , address=104.36.82.7:4911 , piece_id=S7NKW4XURD4AXLWR4MCU34ZMON3QNQPN7AT2LI573QLLLJ6TUQBQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413322687 (376h30m44.296736315s) [noise=true , address=104.36.82.3:4767 , piece_id=34FROCZZSLC7RK4XGN7EZX2N2YDJJWEEYKJOP674GR2Z4JWBTVUQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:41:44.413452023 (376h30m44.296865635s) [noise=true , address=104.36.82.4:4801 , piece_id=ETME2T5RJAAVR4WIM5Y3TTARWURQZXZDGP4LXHEQPSUD5GZPAEKA, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 139.59.144.22:54328->104.36.82.4:4801: use of closed network connection; context deadline exceeded Jul 9 01:41:44.413471848 (376h30m44.296885488s) [noise=true , address=104.36.82.8:4969 , piece_id=E3NOPBM4RLETB4A3ZRHN7ISCCFTM37BHFL7XK7QUJGMG6DIA65PQ, seg_pos={0 0}, upload=301082]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:50846->104.36.82.8:4969: use of closed network connection Jul 9 01:41:44.413539946 (376h30m44.29695356s) [seg_pos={0 0}, upload=301082]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:41:44.577747284 (376h30m44.461160908s) [upload=301082]: Done uploading segments... Jul 9 01:41:44.577784030 (376h30m44.461197666s) [upload=301082]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.166994794s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 28 out of 60 is failed (threshold=3) [........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XXXX] duration: 1m0.001433199s FAIL (2025-07-09 01:41:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001445943s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [......X.X.XXXXX.......XX....................X..X..X.........] duration: 22.93014239s FAIL (2025-07-09 01:41:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 22.93016594s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 27 out of 60 is failed (threshold=3) [.........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XXX] duration: 1m0.000788199s FAIL (2025-07-09 01:40:37 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00081975s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X......X.X.XXXXX.......XX....................X..X..X........] duration: 3.063638346s FAIL (2025-07-09 01:40:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.063670155s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.469819693s FAIL (2025-07-09 01:40:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.469921438s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X......X.X.XXXXX.......XX....................X..X..X.......] duration: 27.835297854s FAIL (2025-07-09 01:39:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 27.835325461s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 26 out of 60 is failed (threshold=3) [..........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.XX] duration: 1m0.000564438s FAIL (2025-07-09 01:39:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000589691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.295849925s FAIL (2025-07-09 01:39:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.295883394s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.X......X.X.XXXXX.......XX....................X..X..X......] duration: 25.483940332s FAIL (2025-07-09 01:38:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.484136634s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 25 out of 60 is failed (threshold=3) [...........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.X] duration: 1m0.000565942s FAIL (2025-07-09 01:38:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000612055s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 2.403786919s FAIL (2025-07-09 01:38:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.403836716s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 2.344426838s FAIL (2025-07-09 01:37:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.344457656s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 25 out of 60 is failed (threshold=3) [X...........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX.] duration: 46.054409217s FAIL (2025-07-09 01:37:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 46.054436027s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.X.X......X.X.XXXXX.......XX....................X..X..X.....] duration: 4.240055387s FAIL (2025-07-09 01:37:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.240129721s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [X.X.X......X.X.XXXXX.......XX....................X..X..X....] duration: 2.618912724s FAIL (2025-07-09 01:36:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.618938184s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 15.231001042s FAIL (2025-07-09 01:36:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 15.231031296s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 25 out of 60 is failed (threshold=3) [.X...........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XXX] duration: 1m0.001894752s FAIL (2025-07-09 01:36:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00194385s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [XX.X.X......X.X.XXXXX.......XX....................X..X..X...] duration: 20.743853877s FAIL (2025-07-09 01:36:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 20.74388414s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.483893597s FAIL (2025-07-09 01:35:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.483921984s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 25 out of 60 is failed (threshold=3) [X.X...........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.XX] duration: 1m0.001867054s FAIL (2025-07-09 01:35:21 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001886982s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.XX.X.X......X.X.XXXXX.......XX....................X..X..X..] duration: 20.90780575s FAIL (2025-07-09 01:35:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 20.907828121s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.294937895s FAIL (2025-07-09 01:34:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.294976959s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.36815053s FAIL (2025-07-09 01:34:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.368181156s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..XX.X.X......X.X.XXXXX.......XX....................X..X..X.] duration: 2.733723196s FAIL (2025-07-09 01:34:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.733759707s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 25 out of 60 is failed (threshold=3) [XX.X...........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.X] duration: 1m0.001603577s FAIL (2025-07-09 01:34:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001632503s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 11.430970913s FAIL (2025-07-09 01:34:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 11.431005242s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [...XX.X.X......X.X.XXXXX.......XX....................X..X..X] duration: 30.000507582s FAIL (2025-07-09 01:34:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000533703s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....XX.X.X......X.X.XXXXX.......XX....................X..X..] duration: 7.714821883s FAIL (2025-07-09 01:33:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.714839439s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.29162088s FAIL (2025-07-09 01:32:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.291653056s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....XX.X.X......X.X.XXXXX.......XX....................X..X.] duration: 22.202680186s FAIL (2025-07-09 01:32:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 22.202983698s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 24 out of 60 is failed (threshold=3) [.XX.X...........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X.] duration: 13.548437566s FAIL (2025-07-09 01:32:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.54845226s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 24 out of 60 is failed (threshold=3) [..XX.X...........XXXXX...X.........X..X...XXX..XXXX.XXXXX..X] duration: 1m0.000969375s FAIL (2025-07-09 01:32:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000990929s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.490816594s FAIL (2025-07-09 01:32:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== cessful: 13 optimal: 13 errs: <nil> Jul 9 01:12:51.706672230 (376h1m51.590085861s) [upload=300705]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 9 01:12:51.706677984 (376h1m51.590091617s) [upload=300705]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 9 01:12:51.883113268 (376h1m51.76652692s) [upload=300705]: Done uploading segments... Jul 9 01:12:51.883135264 (376h1m51.766548909s) [upload=300705]: Upload finished. err: <nil> Jul 9 01:13:20.653341358 (376h2m20.536754991s) [upload=300711]: Starting upload Jul 9 01:13:20.653504915 (376h2m20.536918556s) [upload=300711]: Done starting upload Jul 9 01:13:20.653690252 (376h2m20.537103868s) [upload=300711]: Uploading segments... Jul 9 01:13:20.653990712 (376h2m20.537404332s) [upload=300711]: Got next segment. Inline: false Jul 9 01:13:20.654059122 (376h2m20.537472753s) [upload=300711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 9 01:13:20.654065941 (376h2m20.537479574s) [upload=300711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 9 01:13:20.769797764 (376h2m20.653211413s) [seg_pos={0 0}, upload=300711]: Begin upload segment... Jul 9 01:13:20.769876747 (376h2m20.653290386s) [seg_pos={0 0}, upload=300711]: Done begin upload segment. Jul 9 01:13:20.769954018 (376h2m20.653367648s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770052187 (376h2m20.653465935s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770659155 (376h2m20.6540728s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.770677751 (376h2m20.654091394s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771126425 (376h2m20.654540056s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771277890 (376h2m20.654691526s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771476972 (376h2m20.654890603s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771804253 (376h2m20.655217892s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771829179 (376h2m20.655242816s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.771909583 (376h2m20.655323221s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.772476215 (376h2m20.655889855s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.773678060 (376h2m20.657091699s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774188246 (376h2m20.657601884s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.774573193 (376h2m20.657986824s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775340796 (376h2m20.658754434s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:20.775786189 (376h2m20.659199809s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Uploading piece... Jul 9 01:13:21.378914824 (376h2m21.262328447s) [upload=300711]: Next returned nil segment Jul 9 01:13:21.378938224 (376h2m21.262351854s) [upload=300711]: Waiting for error group managing segments... Jul 9 01:13:21.517933925 (376h2m21.401347538s) [noise=true , address=45.91.116.14:4191 , piece_id=VEYEZNNPLRQSBSE62QVBT2Q4WPKXG4PGDXH5HJKTPVHQQDB4MKPA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.518663136 (376h2m21.402076749s) [noise=true , address=45.91.116.5:4297 , piece_id=R2Z5M6C32BGS2MCKBKMSNTIDLYG5RHCF5CTRNWPQTOWI4PY4B2MQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.522226581 (376h2m21.40564021s) [noise=true , address=45.91.116.8:4047 , piece_id=Z6ZRHEM6QQV4TW4ZHK6NL4Z47NI6IWG7TUH27QSS2L6PTW32YHFA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.535639134 (376h2m21.419052751s) [noise=true , address=45.91.116.8:4048 , piece_id=DOCCMTZC4IEFCVSBXIPSA4F6EIBQ7ZJKGSRLNW5VFE4VODCIWDYA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.539204781 (376h2m21.422618407s) [noise=true , address=45.91.116.7:4011 , piece_id=UY4HNXEUHJDV2UGJZFWEW6EN2MRMSMY3RHOVBEKGK57GFHRXF2LQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.545311795 (376h2m21.428725407s) [noise=true , address=77.90.31.11:4554 , piece_id=GOGCT6WPTDUBGCQ6R7MP7F4SRHMUMKX4DVBLTUBHVI6AHMF25M6A, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548650688 (376h2m21.432064308s) [noise=true , address=45.91.116.6:4975 , piece_id=5UBUFVKBEZA25VPY7XGITKY3THJPJ3V5JWYENF2QUNF4EC7NK7UQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.548766114 (376h2m21.432179732s) [noise=true , address=45.91.116.12:4226 , piece_id=HGFAMZDUY3ML6KVAR4JYAIRQNO5N6Y2ZYH3MLO5NZ5H3J7LBDWUA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.552618914 (376h2m21.436032531s) [noise=true , address=77.90.31.16:4733 , piece_id=IR6MHNY3MOXTYIAFLQAGXS2TU2OPDYJHKAMGN5QY7WIZP5UGCW4Q, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556234201 (376h2m21.439647819s) [noise=true , address=77.90.31.17:4771 , piece_id=XXZHZKKCF6N7TEJXKYQWKFWYJQCCXPDVJQKJ6IG7GYQKMFM4CLSA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.556714631 (376h2m21.440128256s) [noise=true , address=77.90.31.10:4520 , piece_id=4G4JPBIA3BKXM7CTKDF3KMSDX5KEDDDXCNCMMEPWJW2JRZUHUARA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:21.584297058 (376h2m21.467710677s) [noise=true , address=45.91.116.9:4261 , piece_id=34XG5S3MF77RT6W7VRFI5SKX67OLWSFQ3VXT5ECQSQRWQH7LWPHQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: <nil> Jul 9 01:13:50.371498503 (376h2m50.254912119s) [noise=true , address=104.36.82.11:4113 , piece_id=ANKTMIW4IJ4A3F7ALG3FF4NKBQTSPOZ77NLZYE3OFXMZMODPIJCQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: expected piece hash; context deadline exceeded; EOF Jul 9 01:13:50.371544262 (376h2m50.254957891s) [noise=true , address=104.36.82.7:4909 , piece_id=ONV4Q6GJHTEX4N6GHLAYDVXPSYJFRW7ZRECICSZGRRFQUMJOQ7SQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:37278->104.36.82.7:4909: use of closed network connection Jul 9 01:13:50.371667019 (376h2m50.255080637s) [noise=true , address=104.36.82.15:4257 , piece_id=W7UXNP7P4FHL7QTGQTHSKLHUER4Y7W7X3MA7CWJAEYUGTEGY56NA, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:33616->104.36.82.15:4257: use of closed network connection Jul 9 01:13:50.371684908 (376h2m50.255098535s) [noise=true , address=104.36.82.10:4078 , piece_id=3M6PXVRYRN4GZJVHDMIHLZZ6LM53B6NNLX6NJDOGFX7VSZ246CZQ, seg_pos={0 0}, upload=300711]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:57754->104.36.82.10:4078: use of closed network connection Jul 9 01:13:50.371707098 (376h2m50.255120766s) [seg_pos={0 0}, upload=300711]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 9 01:13:50.484313152 (376h2m50.367726771s) [upload=300711]: Done uploading segments... Jul 9 01:13:50.484347418 (376h2m50.367761038s) [upload=300711]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.490849547s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)