Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................X.......................X.X...........] duration: 867.34671ms FAIL (2025-07-27 00:21:45 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 867.387411ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.X..XXX....XXX...X....X.......XX...X....X...XX....X......XXX] duration: 30.000449149s FAIL (2025-07-27 00:21:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000465615s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 36 out of 60 is failed (threshold=3) [XXXXXXXXXXXXX..X.X.X.XX.XXXXX..XX.XX..XXXX.X...........X.XXX] duration: 30.104138693s FAIL (2025-07-27 00:21:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== use of closed network connection Jul 27 00:21:09.467872631 (60h45m40.658171222s) [seg_pos={0 0}, upload=58232]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 27 00:21:09.592234496 (60h45m40.782533081s) [upload=58232]: Done uploading segments... Jul 27 00:21:09.592269495 (60h45m40.782568083s) [upload=58232]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Jul 27 00:21:11.970959896 (60h45m43.161258466s) [upload=58242]: Starting upload Jul 27 00:21:11.971024195 (60h45m43.161322775s) [upload=58242]: Done starting upload Jul 27 00:21:11.971139003 (60h45m43.161437591s) [upload=58242]: Uploading segments... Jul 27 00:21:11.971302704 (60h45m43.161601273s) [upload=58242]: Got next segment. Inline: false Jul 27 00:21:11.971343206 (60h45m43.161641786s) [upload=58242]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:21:11.971348535 (60h45m43.161647104s) [upload=58242]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:21:12.077448274 (60h45m43.267746851s) [seg_pos={0 0}, upload=58242]: Begin upload segment... Jul 27 00:21:12.077533585 (60h45m43.267832154s) [seg_pos={0 0}, upload=58242]: Done begin upload segment. Jul 27 00:21:12.077577633 (60h45m43.267876221s) [noise=true , address=45.91.116.7:4011 , piece_id=UXOM2LUCRAXPCVVTXYM2CR3RF3NVZJY5VT5F6JMWLFL25YQA4KHQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.077590899 (60h45m43.267889488s) [noise=true , address=45.91.116.9:4263 , piece_id=NCJY5GZADHSD3G2MPA24IM7MXZUCBSXKDCJQT2U6QNFRJBRKMXWQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.078090306 (60h45m43.268388885s) [noise=true , address=45.91.116.6:4974 , piece_id=EYCR6JNBHD62IBJ3EOKNPTXH4XYW3NMJGXF3VCWCM2DALKPS3EIA, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.078122091 (60h45m43.26842068s) [noise=true , address=45.91.116.12:4226 , piece_id=Q6L4BUA4ZEIQM4XSQBAABTW4567MM3STFOUJI5ETV3USR4U2ISNQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.078547092 (60h45m43.268845673s) [noise=true , address=45.91.116.8:4045 , piece_id=H55SB34YMETXYCBJTD6KBCMUDPDK56BOVH44INVJWG7C74DP3ICQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.079055053 (60h45m43.269353643s) [noise=true , address=77.90.31.14:4662 , piece_id=YLUE6Z6NGHYTARO5TQ4CLRVKS23HR5BBQE3UWHSGKN2IPBUKCSWA, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.079469481 (60h45m43.269768071s) [noise=true , address=45.91.116.14:4191 , piece_id=4WR26URE2THIH2KLUGEVB2ELF6RKRS4DQGHDIEC22M2LSZSQUDIA, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.079652980 (60h45m43.269951551s) [noise=true , address=45.91.116.4:4939 , piece_id=J6V5UCESRQJMLKKVA25OS36AYGWJFA5HCEBGX27OFWPEBQ5R7WJA, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.080044100 (60h45m43.270342679s) [noise=true , address=104.36.82.5:4839 , piece_id=5PDE2OLSJZIOZRS4WBQ2WLMAN4DASVIQDGUBBI3SV4YIFAKNQNHQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.080440154 (60h45m43.270738744s) [noise=true , address=45.91.116.12:4225 , piece_id=CUMP64FJSU2QTQ343JCW26RTGWF6GP627KSG4L4SYLVML4JLJJUQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.080761328 (60h45m43.271059909s) [noise=true , address=104.36.82.4:4803 , piece_id=S5N75LU63LFDLSJN4FLW6AGPW3JQUQG3Y2QTVWNA2XN2743CLMYQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.081045795 (60h45m43.271344387s) [noise=true , address=77.90.31.10:4517 , piece_id=OCKZITCEILARMZ43WSVZEFFBVXQT5L2I2TL4RBMV3LVNX5FES2UA, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.081548629 (60h45m43.271847219s) [noise=true , address=45.91.116.11:4118 , piece_id=4LB2CZM5STPA2LMIS3UK5PM6PYEAKVNJPS6VLTVXWD6JSFII43NQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.082039650 (60h45m43.27233824s) [noise=true , address=104.36.82.3:4767 , piece_id=LW5YCMDSMLRF7KSABDYTJJNWMI7DEPQI2SQGERNQWN5SWP2VVPVQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.082415151 (60h45m43.272713739s) [noise=true , address=45.91.116.14:4189 , piece_id=TOKDSMVC6CQVAE6GHOTU6ENCZ436WJURDWIHJYLAI66AGA5LSTWQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.082552472 (60h45m43.272851056s) [noise=true , address=45.91.116.6:4975 , piece_id=XHQQ4JDCGWIACRA33CJ4476RTKEXB4FTZTAVODLIN535GJLGL2NQ, seg_pos={0 0}, upload=58242]: Uploading piece... Jul 27 00:21:12.683934702 (60h45m43.874233291s) [upload=58242]: Next returned nil segment Jul 27 00:21:12.683956207 (60h45m43.874254797s) [upload=58242]: Waiting for error group managing segments... Jul 27 00:21:12.820413142 (60h45m44.010711728s) [noise=true , address=45.91.116.7:4011 , piece_id=UXOM2LUCRAXPCVVTXYM2CR3RF3NVZJY5VT5F6JMWLFL25YQA4KHQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.825887451 (60h45m44.016186052s) [noise=true , address=45.91.116.6:4975 , piece_id=XHQQ4JDCGWIACRA33CJ4476RTKEXB4FTZTAVODLIN535GJLGL2NQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.826451332 (60h45m44.01674991s) [noise=true , address=45.91.116.4:4939 , piece_id=J6V5UCESRQJMLKKVA25OS36AYGWJFA5HCEBGX27OFWPEBQ5R7WJA, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.830379260 (60h45m44.02067783s) [noise=true , address=45.91.116.8:4045 , piece_id=H55SB34YMETXYCBJTD6KBCMUDPDK56BOVH44INVJWG7C74DP3ICQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.833784693 (60h45m44.024083282s) [noise=true , address=77.90.31.14:4662 , piece_id=YLUE6Z6NGHYTARO5TQ4CLRVKS23HR5BBQE3UWHSGKN2IPBUKCSWA, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.835535597 (60h45m44.025834186s) [noise=true , address=45.91.116.9:4263 , piece_id=NCJY5GZADHSD3G2MPA24IM7MXZUCBSXKDCJQT2U6QNFRJBRKMXWQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.837833959 (60h45m44.028132529s) [noise=true , address=45.91.116.12:4226 , piece_id=Q6L4BUA4ZEIQM4XSQBAABTW4567MM3STFOUJI5ETV3USR4U2ISNQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.844773077 (60h45m44.035071651s) [noise=true , address=45.91.116.11:4118 , piece_id=4LB2CZM5STPA2LMIS3UK5PM6PYEAKVNJPS6VLTVXWD6JSFII43NQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.845986072 (60h45m44.036284642s) [noise=true , address=45.91.116.12:4225 , piece_id=CUMP64FJSU2QTQ343JCW26RTGWF6GP627KSG4L4SYLVML4JLJJUQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.858863646 (60h45m44.049162221s) [noise=true , address=45.91.116.6:4974 , piece_id=EYCR6JNBHD62IBJ3EOKNPTXH4XYW3NMJGXF3VCWCM2DALKPS3EIA, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.892292666 (60h45m44.082591252s) [noise=true , address=45.91.116.14:4191 , piece_id=4WR26URE2THIH2KLUGEVB2ELF6RKRS4DQGHDIEC22M2LSZSQUDIA, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:12.946648256 (60h45m44.136946837s) [noise=true , address=45.91.116.14:4189 , piece_id=TOKDSMVC6CQVAE6GHOTU6ENCZ436WJURDWIHJYLAI66AGA5LSTWQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: <nil> Jul 27 00:21:41.667426694 (60h46m12.857725273s) [noise=true , address=104.36.82.3:4767 , piece_id=LW5YCMDSMLRF7KSABDYTJJNWMI7DEPQI2SQGERNQWN5SWP2VVPVQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:36236->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:36236->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:36236->104.36.82.3:4767: use of closed network connection Jul 27 00:21:41.667504168 (60h46m12.857802748s) [noise=true , address=77.90.31.10:4517 , piece_id=OCKZITCEILARMZ43WSVZEFFBVXQT5L2I2TL4RBMV3LVNX5FES2UA, seg_pos={0 0}, upload=58242]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:48008->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:21:41.667515702 (60h46m12.857814298s) [noise=true , address=104.36.82.4:4803 , piece_id=S5N75LU63LFDLSJN4FLW6AGPW3JQUQG3Y2QTVWNA2XN2743CLMYQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 139.59.144.22:52976->104.36.82.4:4803: use of closed network connection; write tcp 139.59.144.22:52976->104.36.82.4:4803: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:52976->104.36.82.4:4803: use of closed network connection Jul 27 00:21:41.667618697 (60h46m12.857917284s) [noise=true , address=104.36.82.5:4839 , piece_id=5PDE2OLSJZIOZRS4WBQ2WLMAN4DASVIQDGUBBI3SV4YIFAKNQNHQ, seg_pos={0 0}, upload=58242]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: write tcp 139.59.144.22:46110->104.36.82.5:4839: use of closed network connection; write tcp 139.59.144.22:46110->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:46110->104.36.82.5:4839: use of closed network connection Jul 27 00:21:41.667674840 (60h46m12.857973428s) [seg_pos={0 0}, upload=58242]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 27 00:21:41.769980151 (60h46m12.960278723s) [upload=58242]: Done uploading segments... Jul 27 00:21:41.770008517 (60h46m12.960307087s) [upload=58242]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.104160636s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [...XX.X....X......X..X............X.X..X.X..................] duration: 14.572084705s FAIL (2025-07-27 00:21:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.572142588s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [....XX.X....X......X..X............X.X..X.X.................] duration: 40.650356334s FAIL (2025-07-27 00:21:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 40.650402569s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..X..XXX....XXX...X....X.......XX...X....X...XX....X......XX] duration: 30.001292315s FAIL (2025-07-27 00:21:14 UTC) orbiter warning: context deadline exceeded Duration: 30.00132987s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 36 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXX..X.X.X.XX.XXXXX..XX.XX..XXXX.X...........X.XX] duration: 30.127277063s FAIL (2025-07-27 00:21:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 60h45m11.044982971s) [seg_pos={0 0}, upload=58232]: Begin upload segment... Jul 27 00:20:39.854778522 (60h45m11.045077111s) [noise=true , address=77.90.31.18:4805 , piece_id=XHHJBUOA5KS2C77KMT2SDFBBENU6FRV35K6SIABQZ5U6H5XAVYBA, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.855351953 (60h45m11.045650543s) [noise=true , address=77.90.31.14:4663 , piece_id=DWVFC2PPIQXDDQDLNCPU5COMIDXFNZ7BJGNLKO3UA3QVUAZ5BPCQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.855464994 (60h45m11.045763574s) [seg_pos={0 0}, upload=58232]: Done begin upload segment. Jul 27 00:20:39.855492072 (60h45m11.04579066s) [noise=true , address=45.91.116.5:4298 , piece_id=3TVHULAKANPAZMIWURU5XT47DYCHL4ZHNEFAA5PR4C3RMAYWOHLA, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.856172450 (60h45m11.046471032s) [noise=true , address=45.91.116.14:4190 , piece_id=3AVQIFTKN3AZOKHAFNPUWDTZQ4JLP5AMHD2NRKWJYPZBNZNSCTLQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.856374170 (60h45m11.04667275s) [noise=true , address=104.36.82.3:4767 , piece_id=DZ6UYLBSGDQJ4QOGFLZTW3LHTWVLOLM4IX3X2MROJL4AZE74QXMA, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.856804729 (60h45m11.047103311s) [noise=true , address=77.90.31.21:4878 , piece_id=FPI6GGJUWSKC2JM3V63GTCINGBRT64ZQ4KO2H4T52EEYGG6OFD7A, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.857363341 (60h45m11.047661951s) [noise=true , address=77.90.31.10:4520 , piece_id=73TIHSXZCHZFIKHV6FHHNPRTM5CFUMKKKI7QRRARZXZLFOCIDBQQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.858275988 (60h45m11.048574569s) [noise=true , address=77.90.31.15:4697 , piece_id=C5UTIOUYYGAENDIYACLURCXJNCHYZ5NAOXV3TJC6Z7Y234TPZQZQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.858672731 (60h45m11.048971313s) [noise=true , address=45.91.116.8:4045 , piece_id=QBKXH2RDYUJZY5UXJZ2MI2KRGFHKPE6K3UTH5Q35Y4ZTNVIVHDVA, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.859081730 (60h45m11.049380309s) [noise=true , address=77.90.31.17:4770 , piece_id=BIM6LW7IN7MYN4A3AZN3ZAGNYU5ZFXJJOSD33BIY7GQG4WV3KJPQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.859993176 (60h45m11.050291762s) [noise=true , address=77.90.31.17:4771 , piece_id=B3WVLJ27EORYDXM7VLDX364KWSQYO7TWAXO5E272HL3B74FZHDEQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.860489536 (60h45m11.050788122s) [noise=true , address=104.36.82.8:4972 , piece_id=SCSNQQHXCKI7VOTP7FNVPGKK6YVZM7VIFXBMKPRRY2VUP45H3Q4Q, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.860962396 (60h45m11.051260984s) [noise=true , address=104.36.82.15:4257 , piece_id=PLP7U3UZIFMHFJTLTVMWFPUZMAKO6DJPI4WB5JU56UOIQLJFGWOQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.862170555 (60h45m11.052469137s) [noise=true , address=104.36.82.5:4838 , piece_id=PWM7KU3MLKKLV4Q5HOMWF2YG5HLHMP4KJMUTA6A6IYRHQKTAINSQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.862522851 (60h45m11.052821438s) [noise=true , address=45.91.116.5:4299 , piece_id=PSYWOUGG23SNWNQPK4DJHKZZ7KIVEGYZ2GH6FG4CGEIAO5H3S6NA, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:39.862858690 (60h45m11.053157279s) [noise=true , address=77.90.31.16:4733 , piece_id=NUYF54T6QDCAMWJDB4GTVWEAA2M5BJABRQX7XBYLLCNCTDDIOEOQ, seg_pos={0 0}, upload=58232]: Uploading piece... Jul 27 00:20:40.466360860 (60h45m11.656659429s) [upload=58232]: Next returned nil segment Jul 27 00:20:40.466377614 (60h45m11.656676197s) [upload=58232]: Waiting for error group managing segments... Jul 27 00:20:40.601360537 (60h45m11.791659171s) [noise=true , address=77.90.31.18:4805 , piece_id=XHHJBUOA5KS2C77KMT2SDFBBENU6FRV35K6SIABQZ5U6H5XAVYBA, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:20:40.604299192 (60h45m11.794597776s) [noise=true , address=77.90.31.14:4663 , piece_id=DWVFC2PPIQXDDQDLNCPU5COMIDXFNZ7BJGNLKO3UA3QVUAZ5BPCQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:20:40.606322553 (60h45m11.796621132s) [noise=true , address=45.91.116.5:4298 , piece_id=3TVHULAKANPAZMIWURU5XT47DYCHL4ZHNEFAA5PR4C3RMAYWOHLA, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:20:40.612673145 (60h45m11.802971713s) [noise=true , address=45.91.116.14:4190 , piece_id=3AVQIFTKN3AZOKHAFNPUWDTZQ4JLP5AMHD2NRKWJYPZBNZNSCTLQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:20:40.613187516 (60h45m11.803486087s) [noise=true , address=45.91.116.8:4045 , piece_id=QBKXH2RDYUJZY5UXJZ2MI2KRGFHKPE6K3UTH5Q35Y4ZTNVIVHDVA, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:20:40.625292095 (60h45m11.81559066s) [noise=true , address=45.91.116.5:4299 , piece_id=PSYWOUGG23SNWNQPK4DJHKZZ7KIVEGYZ2GH6FG4CGEIAO5H3S6NA, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:20:40.640257435 (60h45m11.830556011s) [noise=true , address=77.90.31.15:4697 , piece_id=C5UTIOUYYGAENDIYACLURCXJNCHYZ5NAOXV3TJC6Z7Y234TPZQZQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:20:40.727513718 (60h45m11.917812291s) [noise=true , address=77.90.31.16:4733 , piece_id=NUYF54T6QDCAMWJDB4GTVWEAA2M5BJABRQX7XBYLLCNCTDDIOEOQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: <nil> Jul 27 00:21:09.467121476 (60h45m40.657420062s) [noise=true , address=104.36.82.15:4257 , piece_id=PLP7U3UZIFMHFJTLTVMWFPUZMAKO6DJPI4WB5JU56UOIQLJFGWOQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 139.59.144.22:58800->104.36.82.15:4257: use of closed network connection; write tcp 139.59.144.22:58800->104.36.82.15:4257: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:58800->104.36.82.15:4257: use of closed network connection Jul 27 00:21:09.467300495 (60h45m40.65759908s) [noise=true , address=104.36.82.5:4838 , piece_id=PWM7KU3MLKKLV4Q5HOMWF2YG5HLHMP4KJMUTA6A6IYRHQKTAINSQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 139.59.144.22:45600->104.36.82.5:4838: use of closed network connection; write tcp 139.59.144.22:45600->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:45600->104.36.82.5:4838: use of closed network connection Jul 27 00:21:09.467417690 (60h45m40.657716276s) [noise=true , address=104.36.82.3:4767 , piece_id=DZ6UYLBSGDQJ4QOGFLZTW3LHTWVLOLM4IX3X2MROJL4AZE74QXMA, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: write tcp 139.59.144.22:54880->104.36.82.3:4767: use of closed network connection; write tcp 139.59.144.22:54880->104.36.82.3:4767: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54880->104.36.82.3:4767: use of closed network connection Jul 27 00:21:09.467520893 (60h45m40.657819477s) [noise=true , address=77.90.31.21:4878 , piece_id=FPI6GGJUWSKC2JM3V63GTCINGBRT64ZQ4KO2H4T52EEYGG6OFD7A, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:21:09.467583422 (60h45m40.657882006s) [noise=true , address=77.90.31.10:4520 , piece_id=73TIHSXZCHZFIKHV6FHHNPRTM5CFUMKKKI7QRRARZXZLFOCIDBQQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:21:09.467640864 (60h45m40.657939448s) [noise=true , address=77.90.31.17:4771 , piece_id=B3WVLJ27EORYDXM7VLDX364KWSQYO7TWAXO5E272HL3B74FZHDEQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:21:09.467801967 (60h45m40.658100556s) [noise=true , address=77.90.31.17:4770 , piece_id=BIM6LW7IN7MYN4A3AZN3ZAGNYU5ZFXJJOSD33BIY7GQG4WV3KJPQ, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:21:09.467848903 (60h45m40.658147469s) [noise=true , address=104.36.82.8:4972 , piece_id=SCSNQQHXCKI7VOTP7FNVPGKK6YVZM7VIFXBMKPRRY2VUP45H3Q4Q, seg_pos={0 0}, upload=58232]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 139.59.144.22:40202->104.36.82.8:4972: use of closed network connection; write tcp 139.59.144.22:40202->104.36.82.8:4972: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:40202->104.36.82.8:4972: use of closed network connection Jul 27 00:21:09.467872631 (60h45m40.658171222s) [seg_pos={0 0}, upload=58232]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 27 00:21:09.592234496 (60h45m40.782533081s) [upload=58232]: Done uploading segments... Jul 27 00:21:09.592269495 (60h45m40.782568083s) [upload=58232]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.127307919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [X.....XX.X.X......X..X...X.X.X..X..XXX....XX......X..X..XX.X] duration: 1m0.003814118s FAIL (2025-07-27 00:21:05 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== Uploading segments... Jul 27 00:20:06.053785377 (60h44m37.244083953s) [upload=58222]: Got next segment. Inline: false Jul 27 00:20:06.053853298 (60h44m37.244151888s) [upload=58222]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:20:06.053861823 (60h44m37.244160519s) [upload=58222]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:20:06.252366324 (60h44m37.442664907s) [seg_pos={0 0}, upload=58222]: Begin upload segment... Jul 27 00:20:06.252438553 (60h44m37.442737125s) [seg_pos={0 0}, upload=58222]: Done begin upload segment. Jul 27 00:20:06.252606099 (60h44m37.442904684s) [noise=true , address=64.49.13.154:4927 , piece_id=OM3S25NVFX5N32QQDK3F37U7EGSFYLZF76HC6EIXWF6ZYSXKAF4A, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.252642203 (60h44m37.442940862s) [noise=true , address=77.90.31.17:4791 , piece_id=M3EYMJW6NKPJSF5HMNZ6IWKPG7LU4EO7UVVAAQB6UZHFEY5VUXNA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.253507616 (60h44m37.4438062s) [noise=true , address=104.36.82.52:4087 , piece_id=4M6NAXHD57QDLEKR4TQXKRPWWWEIJLF722LG5CXGLCA62XM3LBSA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.255280836 (60h44m37.445579408s) [noise=true , address=104.36.82.47:4551 , piece_id=GUF77677CQY3JKDNTSWNVIOXX5EJHYCMKNR6EGT5VEX4MTKJPIYA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.255567763 (60h44m37.445866347s) [noise=true , address=79.127.219.38:4001 , piece_id=FN7I6JLW763VYOYD4F25L5BNAOSSO66S3K2VKMCUZNUIP5KHP2EA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.255869466 (60h44m37.44616805s) [noise=true , address=77.90.31.16:4761 , piece_id=IMZ4T5MKFWMYE6YTSSVY27PQBYVG4WVE2OG362TTFRUV6XENSR4A, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.256306199 (60h44m37.446604786s) [noise=true , address=77.90.31.12:4599 , piece_id=3JGW6ULAXJSSNLE5H2B7JA7NFU62L6P5QZEPORM7MKEV6A2QAUOA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.256990637 (60h44m37.447289241s) [noise=true , address=79.127.219.47:4048 , piece_id=6KUGGMTFLHQ7CY6LIO6LJXHQQAQTM6HBHQSHUMGIMK4X5UD5ZUFQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.257300348 (60h44m37.447598939s) [noise=true , address=79.127.205.235:4687 , piece_id=QZ2XPCWLSLAEVWWAEW3MDNWQBIKZIWGWM66MS6WFRE5V764RWASQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.257627414 (60h44m37.447925996s) [noise=true , address=79.127.205.234:4603 , piece_id=MZVUAKQ53LQOUTM27EKH33WAKONHJSDCTBHOWVFMTEEHQQIRHCNA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.258037508 (60h44m37.448336101s) [noise=true , address=104.36.82.56:4238 , piece_id=ERJUKXDKBDRW424HZKQFNEERMECCPQPNV3BK4DLG2IVRELPBJFCQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.258436223 (60h44m37.448734818s) [noise=true , address=79.127.201.215:4351 , piece_id=EJVXSDDHINKV5I7V4CDBG7QFNMKRGP42EJYEZQYWD3MPDANGVENQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.259060628 (60h44m37.449359201s) [noise=true , address=77.90.31.11:4569 , piece_id=L5QHGBTZ5DYOIXGCW2COZG62OBYPZZIYZWWEFYXQNM5XXWKH7RLA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.259527812 (60h44m37.449826401s) [noise=true , address=79.127.205.246:4279 , piece_id=ST7JQC6I5DDW54OQFEY2BOYP2Y3CW7TZOKJVAAN3CMY6MZCGPWVQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.260068944 (60h44m37.450367535s) [noise=true , address=104.36.82.46:4493 , piece_id=4LXLBA5RL3QIBJZCZGDGEIDSGRGPBSBBBCIXZ775HPXMNNXXGLNQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.260796213 (60h44m37.451094796s) [noise=true , address=104.36.82.12:4153 , piece_id=SVZ4SLM3JPOMVWNMUIRT42DQ23TNEKVVJBBCLZCC44NOAFTL7HJQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.262217474 (60h44m37.452516059s) [noise=true , address=104.36.82.54:4178 , piece_id=NMVAT4MUHAZQCIMXWNCW6JUX6OOPEG6EYFM7EOBP4GDA7FO4BYJQ, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.262932607 (60h44m37.453231198s) [noise=true , address=45.91.116.9:4277 , piece_id=NSCQ3NWWJB2WRXB6YKXT3FKADCISTWJ4RBUO2CQS7II46YDRUYTA, seg_pos={0 0}, upload=58222]: Uploading piece... Jul 27 00:20:06.675681643 (60h44m37.865980236s) [noise=true , address=79.127.219.47:4048 , piece_id=6KUGGMTFLHQ7CY6LIO6LJXHQQAQTM6HBHQSHUMGIMK4X5UD5ZUFQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: ecclient: upload failed (node:12LWLin5hV2KsxHKkUPMx7p3PDcFGb15CJsvbaGdjSy6wwcgRZL, address:79.127.219.47:4048): protocol: storage node overloaded, request limit: 2000 Jul 27 00:20:06.886327477 (60h44m38.076626059s) [upload=58222]: Next returned nil segment Jul 27 00:20:06.886344820 (60h44m38.07664339s) [upload=58222]: Waiting for error group managing segments... Jul 27 00:20:07.020405288 (60h44m38.21070386s) [noise=true , address=79.127.219.38:4001 , piece_id=FN7I6JLW763VYOYD4F25L5BNAOSSO66S3K2VKMCUZNUIP5KHP2EA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:07.554889945 (60h44m38.745188579s) [noise=true , address=79.127.205.234:4603 , piece_id=MZVUAKQ53LQOUTM27EKH33WAKONHJSDCTBHOWVFMTEEHQQIRHCNA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:07.594400045 (60h44m38.784698627s) [noise=true , address=79.127.205.246:4279 , piece_id=ST7JQC6I5DDW54OQFEY2BOYP2Y3CW7TZOKJVAAN3CMY6MZCGPWVQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:07.633730249 (60h44m38.824028823s) [noise=true , address=79.127.205.235:4687 , piece_id=QZ2XPCWLSLAEVWWAEW3MDNWQBIKZIWGWM66MS6WFRE5V764RWASQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:09.272213458 (60h44m40.462512049s) [noise=true , address=45.91.116.9:4277 , piece_id=NSCQ3NWWJB2WRXB6YKXT3FKADCISTWJ4RBUO2CQS7II46YDRUYTA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:12.246110414 (60h44m43.436408984s) [noise=true , address=79.127.201.215:4351 , piece_id=EJVXSDDHINKV5I7V4CDBG7QFNMKRGP42EJYEZQYWD3MPDANGVENQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:15.417935950 (60h44m46.60823453s) [noise=true , address=64.49.13.154:4927 , piece_id=OM3S25NVFX5N32QQDK3F37U7EGSFYLZF76HC6EIXWF6ZYSXKAF4A, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:19.907576025 (60h44m51.097874602s) [noise=true , address=77.90.31.16:4761 , piece_id=IMZ4T5MKFWMYE6YTSSVY27PQBYVG4WVE2OG362TTFRUV6XENSR4A, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:26.140701164 (60h44m57.330999754s) [noise=true , address=104.36.82.52:4087 , piece_id=4M6NAXHD57QDLEKR4TQXKRPWWWEIJLF722LG5CXGLCA62XM3LBSA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:42.127930277 (60h45m13.318228843s) [noise=true , address=104.36.82.47:4551 , piece_id=GUF77677CQY3JKDNTSWNVIOXX5EJHYCMKNR6EGT5VEX4MTKJPIYA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:44.980030726 (60h45m16.170329305s) [noise=true , address=104.36.82.56:4238 , piece_id=ERJUKXDKBDRW424HZKQFNEERMECCPQPNV3BK4DLG2IVRELPBJFCQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:46.333831111 (60h45m17.524129684s) [noise=true , address=104.36.82.54:4178 , piece_id=NMVAT4MUHAZQCIMXWNCW6JUX6OOPEG6EYFM7EOBP4GDA7FO4BYJQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:20:58.080208598 (60h45m29.270507168s) [noise=true , address=77.90.31.12:4599 , piece_id=3JGW6ULAXJSSNLE5H2B7JA7NFU62L6P5QZEPORM7MKEV6A2QAUOA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:21:02.019052644 (60h45m33.20935123s) [noise=true , address=104.36.82.46:4493 , piece_id=4LXLBA5RL3QIBJZCZGDGEIDSGRGPBSBBBCIXZ775HPXMNNXXGLNQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: <nil> Jul 27 00:21:02.019109795 (60h45m33.209408388s) [seg_pos={0 0}, upload=58222]: Segment reached optimal threshold of 14 pieces. Jul 27 00:21:02.019877909 (60h45m33.210176476s) [noise=true , address=77.90.31.17:4791 , piece_id=M3EYMJW6NKPJSF5HMNZ6IWKPG7LU4EO7UVVAAQB6UZHFEY5VUXNA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM): protocol: write tcp 139.59.144.22:60560->77.90.31.17:4791: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:21:02.020325717 (60h45m33.210624299s) [noise=true , address=77.90.31.11:4569 , piece_id=L5QHGBTZ5DYOIXGCW2COZG62OBYPZZIYZWWEFYXQNM5XXWKH7RLA, seg_pos={0 0}, upload=58222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123jUFZTVjq6adDuvt3cco7dTvwNTQkszZX7CeajMfbeCUmEmuJ): protocol: write tcp 139.59.144.22:57506->77.90.31.11:4569: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:21:02.020946041 (60h45m33.211244625s) [noise=true , address=104.36.82.12:4153 , piece_id=SVZ4SLM3JPOMVWNMUIRT42DQ23TNEKVVJBBCLZCC44NOAFTL7HJQ, seg_pos={0 0}, upload=58222]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124myE1wFhvjA5paJy5tr2hRe8jZCoEdbJLdBwvss1ZnT4GvtZv): protocol: write tcp 139.59.144.22:39458->104.36.82.12:4153: use of closed network connection; context canceled Jul 27 00:21:02.021015246 (60h45m33.211313829s) [seg_pos={0 0}, upload=58222]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 27 00:21:02.021233157 (60h45m33.211531752s) [upload=58222]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:21:02.021241546 (60h45m33.211540132s) [upload=58222]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:21:02.172151181 (60h45m33.362449776s) [upload=58222]: Done uploading segments... Jul 27 00:21:02.172192178 (60h45m33.362490756s) [upload=58222]: Upload finished. err: <nil> Duration: 1m0.003838609s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................X.......................X.X..........] duration: 861.319469ms FAIL (2025-07-27 00:21:05 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 861.361595ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 35 out of 60 is failed (threshold=3) [XXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X.X...X.] duration: 3.797145703s FAIL (2025-07-27 00:20:43 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 3.797190962s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 36 out of 60 is failed (threshold=3) [XXXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X.X...X] duration: 30.139482936s FAIL (2025-07-27 00:20:39 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.139532627s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [.....XX.X....X......X..X............X.X..X.X................] duration: 49.11070548s FAIL (2025-07-27 00:20:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 49.110723451s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................X.......................X.X.........] duration: 944.875698ms FAIL (2025-07-27 00:20:20 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 944.916ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 36 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXX..X.X.X.XX.XXXXX..XX.XX..XXXX.X...........X.X] duration: 30.119122791s FAIL (2025-07-27 00:20:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 4.800948438 (60h43m45.991247027s) [noise=true , address=104.36.82.8:4969 , piece_id=NUHIMUZU66JFBXLA3UK5YAOPOVJWYAVNXVSSFG5EP6ZVBDMFNXIQ, seg_pos={0 0}, upload=58209]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX): protocol: context canceled; context canceled Jul 27 00:19:14.800982140 (60h43m45.991280707s) [seg_pos={0 0}, upload=58209]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 27 00:19:14.801083780 (60h43m45.991382352s) [upload=58209]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:19:14.801090573 (60h43m45.991389156s) [upload=58209]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:19:14.956506171 (60h43m46.146804741s) [upload=58209]: Done uploading segments... Jul 27 00:19:14.956525106 (60h43m46.146823678s) [upload=58209]: Upload finished. err: <nil> Jul 27 00:19:40.396729098 (60h44m11.587027677s) [upload=58215]: Starting upload Jul 27 00:19:40.396793666 (60h44m11.587092246s) [upload=58215]: Done starting upload Jul 27 00:19:40.396996666 (60h44m11.587295234s) [upload=58215]: Uploading segments... Jul 27 00:19:40.397219603 (60h44m11.587518183s) [upload=58215]: Got next segment. Inline: false Jul 27 00:19:40.397273220 (60h44m11.587571799s) [upload=58215]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:19:40.397279181 (60h44m11.587577763s) [upload=58215]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:19:40.504369412 (60h44m11.69466798s) [seg_pos={0 0}, upload=58215]: Begin upload segment... Jul 27 00:19:40.504418700 (60h44m11.694717278s) [seg_pos={0 0}, upload=58215]: Done begin upload segment. Jul 27 00:19:40.504445951 (60h44m11.69474454s) [noise=true , address=77.90.31.10:4519 , piece_id=HSANABEG7J6UNN4KQKAOEU5O2AJHM4CZW3DZIJGC62CQHKJFWP6A, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.504459689 (60h44m11.694758281s) [noise=true , address=104.36.82.6:4874 , piece_id=RDJWS6632PJ7EYOK4LQHAUCCBQUHY672SRLCMSWVUJBUPZMMI6IQ, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.504542510 (60h44m11.694841091s) [noise=true , address=45.91.116.13:4155 , piece_id=OZZE6D3FH3WYGADWIG2DC7DNNXAS2EVNWYTFKJ4J5YS4JNQO3EFA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.504967247 (60h44m11.695265834s) [noise=true , address=45.91.116.5:4298 , piece_id=X5S3JZUX66R3QLX3NPJ6CCH2OF4BK4SEH3UZ7L6YHJ3YH3RACU3Q, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.505044737 (60h44m11.695343322s) [noise=true , address=45.91.116.7:4009 , piece_id=CRHGLLP6DKIT6CJ3MN2L3TQYYDMBAFXR2BEXJ6GMR4ROUYSKMQIA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.505298361 (60h44m11.695596929s) [noise=true , address=45.91.116.9:4262 , piece_id=5XZYTRXABQ65JC72UPLFAIDBPFXFSN2VGWAPQLJ56P5HZIHVBSXA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.505422608 (60h44m11.695721187s) [noise=true , address=77.90.31.11:4555 , piece_id=HJPY3MUMFVPFNR4UGXH2IIUZ2Y4PUHBFUBSTXWSFPOCYYILEFC7A, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.505656683 (60h44m11.695955263s) [noise=true , address=77.90.31.10:4517 , piece_id=X62KJZ3BQZKLVNDJHG5V2IS3DC2RV4HF2FMJTFHLDXF66CRKBLBA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.505975187 (60h44m11.69627377s) [noise=true , address=45.91.116.6:4973 , piece_id=R2LHYJM4MCXJDQVL4LEJTZADRBGN47E2DDDUPM2QNZAL5TS4U3ZA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.506268269 (60h44m11.696566851s) [noise=true , address=45.91.116.4:4940 , piece_id=VL4ESTX5MBS3S2QY2IIJKOUFJUW2O5UZZZTMJ4CHGH64RP6V5QMQ, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.506564049 (60h44m11.696862631s) [noise=true , address=45.91.116.11:4117 , piece_id=4S4JSVPPSPDQUEBBQAJIZH7FJBYOKT37V5RKOLPJLGLJE4SOS3TA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.507482578 (60h44m11.697781164s) [noise=true , address=77.90.31.17:4772 , piece_id=YAJLWSCQNRXTY4WJYYZKV3CH2KR2FBVBBCL2TSNCS2TY6LGMK7KA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.507912011 (60h44m11.698210596s) [noise=true , address=45.91.116.8:4048 , piece_id=QQ224D4VADWHE7UCIHOVSN53MJXCDR43TOAB7D6SOM2YBPSZ2YWQ, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.508320734 (60h44m11.698619313s) [noise=true , address=45.91.116.6:4974 , piece_id=6YBAMWCDNINF6IPKSM4W5BETBS3AQRST3XJBK3G7QK2OTZ2IGOQQ, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.509612141 (60h44m11.699910725s) [noise=true , address=45.91.116.14:4190 , piece_id=62L3MW233SFPO2CX2TD77TX53TJVKGQF3ATZN2BYYZD4XKXCYLWA, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:40.510268028 (60h44m11.700566618s) [noise=true , address=77.90.31.18:4806 , piece_id=ZDIU754NOHL6TESBXYGB5HMEZAUAWD6D3WCTQDSGSEOIG2QFGKQQ, seg_pos={0 0}, upload=58215]: Uploading piece... Jul 27 00:19:41.163563317 (60h44m12.353861909s) [upload=58215]: Next returned nil segment Jul 27 00:19:41.163584204 (60h44m12.353882793s) [upload=58215]: Waiting for error group managing segments... Jul 27 00:19:41.298768713 (60h44m12.489067299s) [noise=true , address=77.90.31.18:4806 , piece_id=ZDIU754NOHL6TESBXYGB5HMEZAUAWD6D3WCTQDSGSEOIG2QFGKQQ, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.299634046 (60h44m12.489932635s) [noise=true , address=45.91.116.13:4155 , piece_id=OZZE6D3FH3WYGADWIG2DC7DNNXAS2EVNWYTFKJ4J5YS4JNQO3EFA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.299729886 (60h44m12.490028452s) [noise=true , address=45.91.116.11:4117 , piece_id=4S4JSVPPSPDQUEBBQAJIZH7FJBYOKT37V5RKOLPJLGLJE4SOS3TA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.300485256 (60h44m12.490783845s) [noise=true , address=45.91.116.6:4973 , piece_id=R2LHYJM4MCXJDQVL4LEJTZADRBGN47E2DDDUPM2QNZAL5TS4U3ZA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.301757626 (60h44m12.492056202s) [noise=true , address=45.91.116.8:4048 , piece_id=QQ224D4VADWHE7UCIHOVSN53MJXCDR43TOAB7D6SOM2YBPSZ2YWQ, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.304383132 (60h44m12.494681718s) [noise=true , address=45.91.116.4:4940 , piece_id=VL4ESTX5MBS3S2QY2IIJKOUFJUW2O5UZZZTMJ4CHGH64RP6V5QMQ, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.305300201 (60h44m12.495598781s) [noise=true , address=45.91.116.5:4298 , piece_id=X5S3JZUX66R3QLX3NPJ6CCH2OF4BK4SEH3UZ7L6YHJ3YH3RACU3Q, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.306049181 (60h44m12.496347752s) [noise=true , address=45.91.116.7:4009 , piece_id=CRHGLLP6DKIT6CJ3MN2L3TQYYDMBAFXR2BEXJ6GMR4ROUYSKMQIA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.309559681 (60h44m12.499858271s) [noise=true , address=45.91.116.6:4974 , piece_id=6YBAMWCDNINF6IPKSM4W5BETBS3AQRST3XJBK3G7QK2OTZ2IGOQQ, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.312448352 (60h44m12.502746935s) [noise=true , address=45.91.116.9:4262 , piece_id=5XZYTRXABQ65JC72UPLFAIDBPFXFSN2VGWAPQLJ56P5HZIHVBSXA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:19:41.460082915 (60h44m12.650381495s) [noise=true , address=45.91.116.14:4190 , piece_id=62L3MW233SFPO2CX2TD77TX53TJVKGQF3ATZN2BYYZD4XKXCYLWA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:20:09.135537408 (60h44m40.325835989s) [noise=true , address=104.36.82.6:4874 , piece_id=RDJWS6632PJ7EYOK4LQHAUCCBQUHY672SRLCMSWVUJBUPZMMI6IQ, seg_pos={0 0}, upload=58215]: Done uploading piece. err: <nil> Jul 27 00:20:10.133113994 (60h44m41.323412581s) [noise=true , address=77.90.31.11:4555 , piece_id=HJPY3MUMFVPFNR4UGXH2IIUZ2Y4PUHBFUBSTXWSFPOCYYILEFC7A, seg_pos={0 0}, upload=58215]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:20:10.135333719 (60h44m41.325632307s) [noise=true , address=77.90.31.17:4772 , piece_id=YAJLWSCQNRXTY4WJYYZKV3CH2KR2FBVBBCL2TSNCS2TY6LGMK7KA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:20:10.135425624 (60h44m41.325724207s) [noise=true , address=77.90.31.10:4519 , piece_id=HSANABEG7J6UNN4KQKAOEU5O2AJHM4CZW3DZIJGC62CQHKJFWP6A, seg_pos={0 0}, upload=58215]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:20:10.135498689 (60h44m41.325797284s) [noise=true , address=77.90.31.10:4517 , piece_id=X62KJZ3BQZKLVNDJHG5V2IS3DC2RV4HF2FMJTFHLDXF66CRKBLBA, seg_pos={0 0}, upload=58215]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 139.59.144.22:53602->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 27 00:20:10.135516888 (60h44m41.325815456s) [seg_pos={0 0}, upload=58215]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 27 00:20:10.244602377 (60h44m41.434900968s) [upload=58215]: Done uploading segments... Jul 27 00:20:10.244653204 (60h44m41.434951796s) [upload=58215]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.119154891s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [.X.....XX.X.X......X..X...X.X.X..X..XXX....XX......X..X..XX.] duration: 38.764864789s FAIL (2025-07-27 00:20:05 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 092 (60h42m57.530764661s) [upload=58198]: Uploading segments... Jul 27 00:18:26.340535285 (60h42m57.53083387s) [upload=58198]: Got next segment. Inline: false Jul 27 00:18:26.340571418 (60h42m57.530869987s) [upload=58198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:18:26.340579596 (60h42m57.530878166s) [upload=58198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:18:26.539846488 (60h42m57.730145084s) [seg_pos={0 0}, upload=58198]: Begin upload segment... Jul 27 00:18:26.539917253 (60h42m57.730215851s) [seg_pos={0 0}, upload=58198]: Done begin upload segment. Jul 27 00:18:26.539960602 (60h42m57.730259194s) [noise=true , address=104.36.82.4:4825 , piece_id=JOI3SHEP3OKDE6S7JIU5W3H4VZAGSCU7JLHQPRDZF5FKI4MH6OXA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540010175 (60h42m57.730308761s) [noise=true , address=79.127.205.234:4615 , piece_id=JLGWAIXQHRDWYYBFBEFY6PCWFLRDL5F6NXKBOQ7EM2AVFDBMHBDA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540099320 (60h42m57.730397902s) [noise=true , address=45.91.116.9:4277 , piece_id=BFV6GM5MUC7TOURD6ELQIUG65TTEL3LQ47GSARRXSKAU66SSVDDQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540540645 (60h42m57.730839225s) [noise=true , address=104.36.82.40:4976 , piece_id=KX5NPCR2KD75OKC3CGVE6FL4M2FUQIRB4VTIDBCFQFIU2Y4P3IIA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540564412 (60h42m57.730863002s) [noise=true , address=79.127.205.246:4287 , piece_id=5TI36RBXMAZLHP5ENOJDE7IR63PBYWUTIJLGQZ3LHVR47XAWB7PQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541072464 (60h42m57.731371052s) [noise=true , address=64.49.13.138:4482 , piece_id=JYXOLSTYUFGDQMEDZ66GOEWMHQLYDCSXNQNCYZI5M66P7U2BE5XA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541085600 (60h42m57.731384182s) [noise=true , address=104.36.82.21:4505 , piece_id=M3GFP5JMUIMICQYWCT7FK4DBBF4E4QORPKRGTBUJKUKLVREXBVVA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541426365 (60h42m57.731724952s) [noise=true , address=79.127.201.215:4350 , piece_id=EEAJAE7VGOTRGXI2YDRPICDIYUPJPSUNO4CRCQSZA6RCGVDMW3IA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541461852 (60h42m57.731760437s) [noise=true , address=104.36.82.46:4510 , piece_id=77XVF4J6JWFUGUNMI4UR7QWIVY3425GFJIEHN2ASZ633AKMBWYAQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541786731 (60h42m57.732085312s) [noise=true , address=77.90.31.15:4703 , piece_id=RCCH3EVNDEABCVHNVQCRPW6O5HLJOT5TLELMYSUMTNHDMCQPZN5Q, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541888958 (60h42m57.732187529s) [noise=true , address=104.36.82.14:4256 , piece_id=W26EI3A3QDMQHDFS6BXLXAKVH22AD6LCTTAGLJ6FY7V7H2VP4LUQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.542166170 (60h42m57.732464758s) [noise=true , address=45.91.116.14:4206 , piece_id=YK5MU6OQYMOSLLA5ZBFZUZK7UG4EQMQSOTR53MBET3NLV3QQ3Z3Q, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.544102862 (60h42m57.734401457s) [noise=true , address=79.127.201.214:4321 , piece_id=72EMNZOQSUZPABYM2VWCSFSJDZVSJPVWXGU3WWFGRXWSKJAQLKRA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.545419837 (60h42m57.735718409s) [noise=true , address=104.36.82.52:4088 , piece_id=MIXJKYOSIVXRGFS5DLMNG6EEDY2TF2BHQZNK3SJOAJXVFDODTMPA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.546535522 (60h42m57.73683411s) [noise=true , address=79.127.205.245:4236 , piece_id=WYHR4LMEZQIUXEGKVMSIYELQGJPWI7XT5JEIG773K6KUUASLMO6Q, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.546773188 (60h42m57.73707176s) [noise=true , address=104.36.82.54:4165 , piece_id=2X3T7M3Q2Q7LYO2M2LQNFNDUIDUENRAF54ZFKZZNUY27RHAOLHXQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.546956308 (60h42m57.737254877s) [noise=true , address=79.127.205.235:4678 , piece_id=RFR7XJIFALYSYKXN7E6NTVDOSIVJP6ILRIKHR2FN2GWMGAIBL22A, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.549017419 (60h42m57.739315986s) [noise=true , address=79.127.219.38:4018 , piece_id=HJKUJRM5ZIYH6R372SBAOGJG6I4JXVSVSHQQLR6ULZ6GA4OTUUNQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:27.076265828 (60h42m58.266564407s) [noise=true , address=79.127.201.214:4321 , piece_id=72EMNZOQSUZPABYM2VWCSFSJDZVSJPVWXGU3WWFGRXWSKJAQLKRA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: ecclient: upload failed (node:127EgxjVRhAGcz1DJjE539rWnmDjsVTRFQc1esEZUyQ1yxGiCD5, address:79.127.201.214:4321): protocol: storage node overloaded, request limit: 2000 Jul 27 00:18:27.114086555 (60h42m58.304385137s) [upload=58198]: Next returned nil segment Jul 27 00:18:27.114106230 (60h42m58.304404809s) [upload=58198]: Waiting for error group managing segments... Jul 27 00:18:27.287224088 (60h42m58.47752266s) [noise=true , address=77.90.31.15:4703 , piece_id=RCCH3EVNDEABCVHNVQCRPW6O5HLJOT5TLELMYSUMTNHDMCQPZN5Q, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.322052884 (60h42m58.512351455s) [noise=true , address=79.127.219.38:4018 , piece_id=HJKUJRM5ZIYH6R372SBAOGJG6I4JXVSVSHQQLR6ULZ6GA4OTUUNQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.371721099 (60h42m58.562019685s) [noise=true , address=45.91.116.14:4206 , piece_id=YK5MU6OQYMOSLLA5ZBFZUZK7UG4EQMQSOTR53MBET3NLV3QQ3Z3Q, seg_pos={0 0}, upload=58198]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: storage node overloaded, request limit: 2000 Jul 27 00:18:27.898818441 (60h42m59.089117028s) [noise=true , address=79.127.205.245:4236 , piece_id=WYHR4LMEZQIUXEGKVMSIYELQGJPWI7XT5JEIG773K6KUUASLMO6Q, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.905631113 (60h42m59.095929693s) [noise=true , address=79.127.205.246:4287 , piece_id=5TI36RBXMAZLHP5ENOJDE7IR63PBYWUTIJLGQZ3LHVR47XAWB7PQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.911342576 (60h42m59.10164117s) [noise=true , address=79.127.205.234:4615 , piece_id=JLGWAIXQHRDWYYBFBEFY6PCWFLRDL5F6NXKBOQ7EM2AVFDBMHBDA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:28.801882448 (60h42m59.99218102s) [noise=true , address=79.127.205.235:4678 , piece_id=RFR7XJIFALYSYKXN7E6NTVDOSIVJP6ILRIKHR2FN2GWMGAIBL22A, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:32.320260291 (60h43m3.510558858s) [noise=true , address=64.49.13.138:4482 , piece_id=JYXOLSTYUFGDQMEDZ66GOEWMHQLYDCSXNQNCYZI5M66P7U2BE5XA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:37.000988157 (60h43m8.191286767s) [noise=true , address=45.91.116.9:4277 , piece_id=BFV6GM5MUC7TOURD6ELQIUG65TTEL3LQ47GSARRXSKAU66SSVDDQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:48.076055008 (60h43m19.26635358s) [noise=true , address=79.127.201.215:4350 , piece_id=EEAJAE7VGOTRGXI2YDRPICDIYUPJPSUNO4CRCQSZA6RCGVDMW3IA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:52.276332667 (60h43m23.46663131s) [noise=true , address=104.36.82.54:4165 , piece_id=2X3T7M3Q2Q7LYO2M2LQNFNDUIDUENRAF54ZFKZZNUY27RHAOLHXQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:02.324945333 (60h43m33.515243916s) [noise=true , address=104.36.82.52:4088 , piece_id=MIXJKYOSIVXRGFS5DLMNG6EEDY2TF2BHQZNK3SJOAJXVFDODTMPA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:04.569319121 (60h43m35.759617697s) [noise=true , address=104.36.82.14:4256 , piece_id=W26EI3A3QDMQHDFS6BXLXAKVH22AD6LCTTAGLJ6FY7V7H2VP4LUQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:08.910658123 (60h43m40.100956695s) [noise=true , address=104.36.82.40:4976 , piece_id=KX5NPCR2KD75OKC3CGVE6FL4M2FUQIRB4VTIDBCFQFIU2Y4P3IIA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:17.906753890 (60h43m49.09705246s) [noise=true , address=104.36.82.46:4510 , piece_id=77XVF4J6JWFUGUNMI4UR7QWIVY3425GFJIEHN2ASZ633AKMBWYAQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:17.906795842 (60h43m49.097094426s) [seg_pos={0 0}, upload=58198]: Segment reached optimal threshold of 14 pieces. Jul 27 00:19:17.907064183 (60h43m49.09736279s) [noise=true , address=104.36.82.21:4505 , piece_id=M3GFP5JMUIMICQYWCT7FK4DBBF4E4QORPKRGTBUJKUKLVREXBVVA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Eh2hcsqTzPZqt3XLXPLcrC2nMzyZMimsQFDCSmzSEheqYn4bF): protocol: expected piece hash; context canceled; EOF Jul 27 00:19:17.907104335 (60h43m49.097402901s) [noise=true , address=104.36.82.4:4825 , piece_id=JOI3SHEP3OKDE6S7JIU5W3H4VZAGSCU7JLHQPRDZF5FKI4MH6OXA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb): protocol: write tcp 139.59.144.22:59624->104.36.82.4:4825: use of closed network connection; write tcp 139.59.144.22:59624->104.36.82.4:4825: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59624->104.36.82.4:4825: use of closed network connection Jul 27 00:19:17.907137475 (60h43m49.097436061s) [seg_pos={0 0}, upload=58198]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 27 00:19:17.907267075 (60h43m49.097565665s) [upload=58198]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:19:17.907274092 (60h43m49.097572687s) [upload=58198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:19:18.066760070 (60h43m49.25705865s) [upload=58198]: Done uploading segments... Jul 27 00:19:18.066782223 (60h43m49.257080799s) [upload=58198]: Upload finished. err: <nil> Duration: 38.764921327s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...X..XXX....XXX...X....X.......XX...X....X...XX....X......X] duration: 30.001267802s FAIL (2025-07-27 00:20:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001293589s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................X.......................X.X........] duration: 853.248246ms FAIL (2025-07-27 00:19:52 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 853.279543ms core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [......XX.X....X......X..X............X.X..X.X...............] duration: 29.02545545s FAIL (2025-07-27 00:19:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 29.02549091s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [..X.....XX.X.X......X..X...X.X.X..X..XXX....XX......X..X..XX] duration: 1m0.006021127s FAIL (2025-07-27 00:19:26 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 092 (60h42m57.530764661s) [upload=58198]: Uploading segments... Jul 27 00:18:26.340535285 (60h42m57.53083387s) [upload=58198]: Got next segment. Inline: false Jul 27 00:18:26.340571418 (60h42m57.530869987s) [upload=58198]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:18:26.340579596 (60h42m57.530878166s) [upload=58198]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:18:26.539846488 (60h42m57.730145084s) [seg_pos={0 0}, upload=58198]: Begin upload segment... Jul 27 00:18:26.539917253 (60h42m57.730215851s) [seg_pos={0 0}, upload=58198]: Done begin upload segment. Jul 27 00:18:26.539960602 (60h42m57.730259194s) [noise=true , address=104.36.82.4:4825 , piece_id=JOI3SHEP3OKDE6S7JIU5W3H4VZAGSCU7JLHQPRDZF5FKI4MH6OXA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540010175 (60h42m57.730308761s) [noise=true , address=79.127.205.234:4615 , piece_id=JLGWAIXQHRDWYYBFBEFY6PCWFLRDL5F6NXKBOQ7EM2AVFDBMHBDA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540099320 (60h42m57.730397902s) [noise=true , address=45.91.116.9:4277 , piece_id=BFV6GM5MUC7TOURD6ELQIUG65TTEL3LQ47GSARRXSKAU66SSVDDQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540540645 (60h42m57.730839225s) [noise=true , address=104.36.82.40:4976 , piece_id=KX5NPCR2KD75OKC3CGVE6FL4M2FUQIRB4VTIDBCFQFIU2Y4P3IIA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.540564412 (60h42m57.730863002s) [noise=true , address=79.127.205.246:4287 , piece_id=5TI36RBXMAZLHP5ENOJDE7IR63PBYWUTIJLGQZ3LHVR47XAWB7PQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541072464 (60h42m57.731371052s) [noise=true , address=64.49.13.138:4482 , piece_id=JYXOLSTYUFGDQMEDZ66GOEWMHQLYDCSXNQNCYZI5M66P7U2BE5XA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541085600 (60h42m57.731384182s) [noise=true , address=104.36.82.21:4505 , piece_id=M3GFP5JMUIMICQYWCT7FK4DBBF4E4QORPKRGTBUJKUKLVREXBVVA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541426365 (60h42m57.731724952s) [noise=true , address=79.127.201.215:4350 , piece_id=EEAJAE7VGOTRGXI2YDRPICDIYUPJPSUNO4CRCQSZA6RCGVDMW3IA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541461852 (60h42m57.731760437s) [noise=true , address=104.36.82.46:4510 , piece_id=77XVF4J6JWFUGUNMI4UR7QWIVY3425GFJIEHN2ASZ633AKMBWYAQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541786731 (60h42m57.732085312s) [noise=true , address=77.90.31.15:4703 , piece_id=RCCH3EVNDEABCVHNVQCRPW6O5HLJOT5TLELMYSUMTNHDMCQPZN5Q, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.541888958 (60h42m57.732187529s) [noise=true , address=104.36.82.14:4256 , piece_id=W26EI3A3QDMQHDFS6BXLXAKVH22AD6LCTTAGLJ6FY7V7H2VP4LUQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.542166170 (60h42m57.732464758s) [noise=true , address=45.91.116.14:4206 , piece_id=YK5MU6OQYMOSLLA5ZBFZUZK7UG4EQMQSOTR53MBET3NLV3QQ3Z3Q, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.544102862 (60h42m57.734401457s) [noise=true , address=79.127.201.214:4321 , piece_id=72EMNZOQSUZPABYM2VWCSFSJDZVSJPVWXGU3WWFGRXWSKJAQLKRA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.545419837 (60h42m57.735718409s) [noise=true , address=104.36.82.52:4088 , piece_id=MIXJKYOSIVXRGFS5DLMNG6EEDY2TF2BHQZNK3SJOAJXVFDODTMPA, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.546535522 (60h42m57.73683411s) [noise=true , address=79.127.205.245:4236 , piece_id=WYHR4LMEZQIUXEGKVMSIYELQGJPWI7XT5JEIG773K6KUUASLMO6Q, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.546773188 (60h42m57.73707176s) [noise=true , address=104.36.82.54:4165 , piece_id=2X3T7M3Q2Q7LYO2M2LQNFNDUIDUENRAF54ZFKZZNUY27RHAOLHXQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.546956308 (60h42m57.737254877s) [noise=true , address=79.127.205.235:4678 , piece_id=RFR7XJIFALYSYKXN7E6NTVDOSIVJP6ILRIKHR2FN2GWMGAIBL22A, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:26.549017419 (60h42m57.739315986s) [noise=true , address=79.127.219.38:4018 , piece_id=HJKUJRM5ZIYH6R372SBAOGJG6I4JXVSVSHQQLR6ULZ6GA4OTUUNQ, seg_pos={0 0}, upload=58198]: Uploading piece... Jul 27 00:18:27.076265828 (60h42m58.266564407s) [noise=true , address=79.127.201.214:4321 , piece_id=72EMNZOQSUZPABYM2VWCSFSJDZVSJPVWXGU3WWFGRXWSKJAQLKRA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: ecclient: upload failed (node:127EgxjVRhAGcz1DJjE539rWnmDjsVTRFQc1esEZUyQ1yxGiCD5, address:79.127.201.214:4321): protocol: storage node overloaded, request limit: 2000 Jul 27 00:18:27.114086555 (60h42m58.304385137s) [upload=58198]: Next returned nil segment Jul 27 00:18:27.114106230 (60h42m58.304404809s) [upload=58198]: Waiting for error group managing segments... Jul 27 00:18:27.287224088 (60h42m58.47752266s) [noise=true , address=77.90.31.15:4703 , piece_id=RCCH3EVNDEABCVHNVQCRPW6O5HLJOT5TLELMYSUMTNHDMCQPZN5Q, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.322052884 (60h42m58.512351455s) [noise=true , address=79.127.219.38:4018 , piece_id=HJKUJRM5ZIYH6R372SBAOGJG6I4JXVSVSHQQLR6ULZ6GA4OTUUNQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.371721099 (60h42m58.562019685s) [noise=true , address=45.91.116.14:4206 , piece_id=YK5MU6OQYMOSLLA5ZBFZUZK7UG4EQMQSOTR53MBET3NLV3QQ3Z3Q, seg_pos={0 0}, upload=58198]: Done uploading piece. err: ecclient: upload failed (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK, address:45.91.116.14:4206): protocol: storage node overloaded, request limit: 2000 Jul 27 00:18:27.898818441 (60h42m59.089117028s) [noise=true , address=79.127.205.245:4236 , piece_id=WYHR4LMEZQIUXEGKVMSIYELQGJPWI7XT5JEIG773K6KUUASLMO6Q, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.905631113 (60h42m59.095929693s) [noise=true , address=79.127.205.246:4287 , piece_id=5TI36RBXMAZLHP5ENOJDE7IR63PBYWUTIJLGQZ3LHVR47XAWB7PQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:27.911342576 (60h42m59.10164117s) [noise=true , address=79.127.205.234:4615 , piece_id=JLGWAIXQHRDWYYBFBEFY6PCWFLRDL5F6NXKBOQ7EM2AVFDBMHBDA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:28.801882448 (60h42m59.99218102s) [noise=true , address=79.127.205.235:4678 , piece_id=RFR7XJIFALYSYKXN7E6NTVDOSIVJP6ILRIKHR2FN2GWMGAIBL22A, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:32.320260291 (60h43m3.510558858s) [noise=true , address=64.49.13.138:4482 , piece_id=JYXOLSTYUFGDQMEDZ66GOEWMHQLYDCSXNQNCYZI5M66P7U2BE5XA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:37.000988157 (60h43m8.191286767s) [noise=true , address=45.91.116.9:4277 , piece_id=BFV6GM5MUC7TOURD6ELQIUG65TTEL3LQ47GSARRXSKAU66SSVDDQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:48.076055008 (60h43m19.26635358s) [noise=true , address=79.127.201.215:4350 , piece_id=EEAJAE7VGOTRGXI2YDRPICDIYUPJPSUNO4CRCQSZA6RCGVDMW3IA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:18:52.276332667 (60h43m23.46663131s) [noise=true , address=104.36.82.54:4165 , piece_id=2X3T7M3Q2Q7LYO2M2LQNFNDUIDUENRAF54ZFKZZNUY27RHAOLHXQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:02.324945333 (60h43m33.515243916s) [noise=true , address=104.36.82.52:4088 , piece_id=MIXJKYOSIVXRGFS5DLMNG6EEDY2TF2BHQZNK3SJOAJXVFDODTMPA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:04.569319121 (60h43m35.759617697s) [noise=true , address=104.36.82.14:4256 , piece_id=W26EI3A3QDMQHDFS6BXLXAKVH22AD6LCTTAGLJ6FY7V7H2VP4LUQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:08.910658123 (60h43m40.100956695s) [noise=true , address=104.36.82.40:4976 , piece_id=KX5NPCR2KD75OKC3CGVE6FL4M2FUQIRB4VTIDBCFQFIU2Y4P3IIA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:17.906753890 (60h43m49.09705246s) [noise=true , address=104.36.82.46:4510 , piece_id=77XVF4J6JWFUGUNMI4UR7QWIVY3425GFJIEHN2ASZ633AKMBWYAQ, seg_pos={0 0}, upload=58198]: Done uploading piece. err: <nil> Jul 27 00:19:17.906795842 (60h43m49.097094426s) [seg_pos={0 0}, upload=58198]: Segment reached optimal threshold of 14 pieces. Jul 27 00:19:17.907064183 (60h43m49.09736279s) [noise=true , address=104.36.82.21:4505 , piece_id=M3GFP5JMUIMICQYWCT7FK4DBBF4E4QORPKRGTBUJKUKLVREXBVVA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Eh2hcsqTzPZqt3XLXPLcrC2nMzyZMimsQFDCSmzSEheqYn4bF): protocol: expected piece hash; context canceled; EOF Jul 27 00:19:17.907104335 (60h43m49.097402901s) [noise=true , address=104.36.82.4:4825 , piece_id=JOI3SHEP3OKDE6S7JIU5W3H4VZAGSCU7JLHQPRDZF5FKI4MH6OXA, seg_pos={0 0}, upload=58198]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb): protocol: write tcp 139.59.144.22:59624->104.36.82.4:4825: use of closed network connection; write tcp 139.59.144.22:59624->104.36.82.4:4825: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:59624->104.36.82.4:4825: use of closed network connection Jul 27 00:19:17.907137475 (60h43m49.097436061s) [seg_pos={0 0}, upload=58198]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 27 00:19:17.907267075 (60h43m49.097565665s) [upload=58198]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:19:17.907274092 (60h43m49.097572687s) [upload=58198]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:19:18.066760070 (60h43m49.25705865s) [upload=58198]: Done uploading segments... Jul 27 00:19:18.066782223 (60h43m49.257080799s) [upload=58198]: Upload finished. err: <nil> Duration: 1m0.006049277s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [....X..XXX....XXX...X....X.......XX...X....X...XX....X......] duration: 3.906725688s FAIL (2025-07-27 00:19:18 UTC) orbiter warning: context deadline exceeded Duration: 3.906757043s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 35 out of 60 is failed (threshold=3) [.XXXXXXXXXXXXXXX..X.X.X.XX.XXXXX..XX.XX..XXXX.X...........X.] duration: 2.596025619s FAIL (2025-07-27 00:19:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:16:54.180647759 (60h41m25.370946338s) [upload=58178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:16:54.413557226 (60h41m25.60385585s) [upload=58178]: Done uploading segments... Jul 27 00:16:54.413584017 (60h41m25.603882603s) [upload=58178]: Upload finished. err: <nil> Jul 27 00:18:03.378950481 (60h42m34.569249069s) [upload=58192]: Starting upload Jul 27 00:18:03.379107545 (60h42m34.569406136s) [upload=58192]: Done starting upload Jul 27 00:18:03.379254221 (60h42m34.569552802s) [upload=58192]: Uploading segments... Jul 27 00:18:03.379718243 (60h42m34.570016834s) [upload=58192]: Got next segment. Inline: false Jul 27 00:18:03.379825295 (60h42m34.570123882s) [upload=58192]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:18:03.379832570 (60h42m34.570131159s) [upload=58192]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:18:03.487173088 (60h42m34.677471668s) [seg_pos={0 0}, upload=58192]: Begin upload segment... Jul 27 00:18:03.487222342 (60h42m34.677520913s) [seg_pos={0 0}, upload=58192]: Done begin upload segment. Jul 27 00:18:03.487254969 (60h42m34.677553554s) [noise=true , address=77.90.31.13:4628 , piece_id=OCTUFUZKUA6SDJN4QVALXPCGYULF75XL73M2BPGXEJTSJSJHLUNQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487287770 (60h42m34.67758637s) [noise=true , address=104.36.82.4:4802 , piece_id=WIO6KTPX43VHXO64HRMO5KGJ7FD2P6DMSAUKEYV5P7R5NH4XRQHQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487637044 (60h42m34.67793563s) [noise=true , address=104.36.82.15:4258 , piece_id=2YWOHD3RSNBD7K24ZD6JJJOERI4MLBMDPRITYCUFKQHRDTH4MYFA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487900358 (60h42m34.678198945s) [noise=true , address=45.91.116.13:4154 , piece_id=KSR45P7YV4HEABWB5CVQM4NFL7MUNKJTGV4JS3CJ6SRZQYXZ2DHA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487931250 (60h42m34.678229831s) [noise=true , address=77.90.31.16:4733 , piece_id=NSGBPILBI3UYR2L57T2Q2W3MUID7J7M6QKPMKZJ7HYUCSEO77WBA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.488296846 (60h42m34.678595427s) [noise=true , address=45.91.116.5:4298 , piece_id=NLMFP5NK6F5ORI754VHNPIDLTKL5BTPY7SES6BFXG4KIOAUZQH6Q, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.488596786 (60h42m34.678895365s) [noise=true , address=45.91.116.7:4011 , piece_id=4BDNHSORHLCE54CCFG6TLY2XNDPHNZ2F6ZIHVS73NVRP553Y3JYQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489061071 (60h42m34.679359654s) [noise=true , address=45.91.116.6:4973 , piece_id=N4HN73VB6DHJ57ERGBADR5OYNGMENYGWTHGXQBZSZSJM5T47PNFQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489376608 (60h42m34.679675187s) [noise=true , address=45.91.116.4:4939 , piece_id=FB6QKW5GMFL2NMRAAYU3A5EBVKE7WUMX6FQAS65AVL726HSL3PMA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489708319 (60h42m34.680006908s) [noise=true , address=45.91.116.6:4975 , piece_id=T24TA7MTX3KUB3DRDJYYPOUEAWIMNK2KHBZPZVZFEAQSQQSUHDAA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489979774 (60h42m34.680278366s) [noise=true , address=104.36.82.9:4007 , piece_id=ZBKT7APSNONYS2ERXUROUVGCXVBW27VVFLSJK4TTR4I725LKOQ4Q, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.490367371 (60h42m34.680665962s) [noise=true , address=45.91.116.12:4225 , piece_id=UGWVN4GYV67MHU4ATDO53763FIH7ZPANW7MCEJLE4CZ75AEZIK7Q, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.490713165 (60h42m34.681011749s) [noise=true , address=77.90.31.15:4699 , piece_id=6VYS4G2C6FWVUYAM2FV74X4NCSPEHPK2R34EJRVUM7PLSSYPDGOQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.491120809 (60h42m34.681419388s) [noise=true , address=77.90.31.12:4589 , piece_id=BGZOTIBEZODJU4Z3WACJBR22TVK4JTVHBRBHULAM42UXVMTXNKFQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.491486611 (60h42m34.681785193s) [noise=true , address=45.91.116.14:4189 , piece_id=2N3RHWIGQISGX6MANQ37RT4GBYZOXWOKPVTLQJDNURO5DM2JV3JQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.491815645 (60h42m34.682114234s) [noise=true , address=104.36.82.6:4873 , piece_id=NOMBRMCMTZGRFGDLFHCEIIIDNU3E3VGIUVRH7AUB2YPZI3NDTNKQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:04.100400461 (60h42m35.29069905s) [upload=58192]: Next returned nil segment Jul 27 00:18:04.100425241 (60h42m35.290723832s) [upload=58192]: Waiting for error group managing segments... Jul 27 00:18:04.237866049 (60h42m35.428164619s) [noise=true , address=77.90.31.15:4699 , piece_id=6VYS4G2C6FWVUYAM2FV74X4NCSPEHPK2R34EJRVUM7PLSSYPDGOQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.243114679 (60h42m35.433413244s) [noise=true , address=45.91.116.4:4939 , piece_id=FB6QKW5GMFL2NMRAAYU3A5EBVKE7WUMX6FQAS65AVL726HSL3PMA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.246965506 (60h42m35.43726409s) [noise=true , address=45.91.116.7:4011 , piece_id=4BDNHSORHLCE54CCFG6TLY2XNDPHNZ2F6ZIHVS73NVRP553Y3JYQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.247104772 (60h42m35.437403355s) [noise=true , address=45.91.116.13:4154 , piece_id=KSR45P7YV4HEABWB5CVQM4NFL7MUNKJTGV4JS3CJ6SRZQYXZ2DHA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.248545042 (60h42m35.43884361s) [noise=true , address=45.91.116.6:4973 , piece_id=N4HN73VB6DHJ57ERGBADR5OYNGMENYGWTHGXQBZSZSJM5T47PNFQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.248681535 (60h42m35.438980103s) [noise=true , address=45.91.116.6:4975 , piece_id=T24TA7MTX3KUB3DRDJYYPOUEAWIMNK2KHBZPZVZFEAQSQQSUHDAA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.254634547 (60h42m35.444933129s) [noise=true , address=45.91.116.5:4298 , piece_id=NLMFP5NK6F5ORI754VHNPIDLTKL5BTPY7SES6BFXG4KIOAUZQH6Q, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.256124428 (60h42m35.446422998s) [noise=true , address=77.90.31.16:4733 , piece_id=NSGBPILBI3UYR2L57T2Q2W3MUID7J7M6QKPMKZJ7HYUCSEO77WBA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.257560649 (60h42m35.447859235s) [noise=true , address=45.91.116.12:4225 , piece_id=UGWVN4GYV67MHU4ATDO53763FIH7ZPANW7MCEJLE4CZ75AEZIK7Q, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.528204477 (60h42m35.718503043s) [noise=true , address=45.91.116.14:4189 , piece_id=2N3RHWIGQISGX6MANQ37RT4GBYZOXWOKPVTLQJDNURO5DM2JV3JQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:05.628152792 (60h42m36.818451377s) [noise=true , address=77.90.31.13:4628 , piece_id=OCTUFUZKUA6SDJN4QVALXPCGYULF75XL73M2BPGXEJTSJSJHLUNQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:05.777962181 (60h42m36.968260754s) [noise=true , address=77.90.31.12:4589 , piece_id=BGZOTIBEZODJU4Z3WACJBR22TVK4JTVHBRBHULAM42UXVMTXNKFQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:33.101710794 (60h43m4.292009373s) [noise=true , address=104.36.82.4:4802 , piece_id=WIO6KTPX43VHXO64HRMO5KGJ7FD2P6DMSAUKEYV5P7R5NH4XRQHQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:41636->104.36.82.4:4802: use of closed network connection; write tcp 139.59.144.22:41636->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41636->104.36.82.4:4802: use of closed network connection Jul 27 00:18:33.101810707 (60h43m4.292109286s) [noise=true , address=104.36.82.9:4007 , piece_id=ZBKT7APSNONYS2ERXUROUVGCXVBW27VVFLSJK4TTR4I725LKOQ4Q, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46944->104.36.82.9:4007: use of closed network connection; write tcp 139.59.144.22:46944->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: context deadline exceeded Jul 27 00:18:33.101886971 (60h43m4.292185551s) [noise=true , address=104.36.82.15:4258 , piece_id=2YWOHD3RSNBD7K24ZD6JJJOERI4MLBMDPRITYCUFKQHRDTH4MYFA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:54986->104.36.82.15:4258: use of closed network connection; write tcp 139.59.144.22:54986->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54986->104.36.82.15:4258: use of closed network connection Jul 27 00:18:33.101992511 (60h43m4.292291099s) [noise=true , address=104.36.82.6:4873 , piece_id=NOMBRMCMTZGRFGDLFHCEIIIDNU3E3VGIUVRH7AUB2YPZI3NDTNKQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: write tcp 139.59.144.22:34762->104.36.82.6:4873: use of closed network connection; write tcp 139.59.144.22:34762->104.36.82.6:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34762->104.36.82.6:4873: use of closed network connection Jul 27 00:18:33.102056717 (60h43m4.292355309s) [seg_pos={0 0}, upload=58192]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 27 00:18:33.203833358 (60h43m4.39413193s) [upload=58192]: Done uploading segments... Jul 27 00:18:33.203860208 (60h43m4.394158787s) [upload=58192]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.596062457s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 36 out of 60 is failed (threshold=3) [XXXXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X.X...] duration: 4.391138805s FAIL (2025-07-27 00:19:08 UTC) orbiter warning: uplink: context deadline exceeded Duration: 4.391175337s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........................X.......................X.X.......] duration: 868.120868ms FAIL (2025-07-27 00:18:42 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 868.184094ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 36 out of 60 is failed (threshold=3) [X.XXXXXXXXXXXXXXX..X.X.X.XX.XXXXX..XX.XX..XXXX.X...........X] duration: 30.104396232s FAIL (2025-07-27 00:18:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:16:54.180647759 (60h41m25.370946338s) [upload=58178]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:16:54.413557226 (60h41m25.60385585s) [upload=58178]: Done uploading segments... Jul 27 00:16:54.413584017 (60h41m25.603882603s) [upload=58178]: Upload finished. err: <nil> Jul 27 00:18:03.378950481 (60h42m34.569249069s) [upload=58192]: Starting upload Jul 27 00:18:03.379107545 (60h42m34.569406136s) [upload=58192]: Done starting upload Jul 27 00:18:03.379254221 (60h42m34.569552802s) [upload=58192]: Uploading segments... Jul 27 00:18:03.379718243 (60h42m34.570016834s) [upload=58192]: Got next segment. Inline: false Jul 27 00:18:03.379825295 (60h42m34.570123882s) [upload=58192]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:18:03.379832570 (60h42m34.570131159s) [upload=58192]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:18:03.487173088 (60h42m34.677471668s) [seg_pos={0 0}, upload=58192]: Begin upload segment... Jul 27 00:18:03.487222342 (60h42m34.677520913s) [seg_pos={0 0}, upload=58192]: Done begin upload segment. Jul 27 00:18:03.487254969 (60h42m34.677553554s) [noise=true , address=77.90.31.13:4628 , piece_id=OCTUFUZKUA6SDJN4QVALXPCGYULF75XL73M2BPGXEJTSJSJHLUNQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487287770 (60h42m34.67758637s) [noise=true , address=104.36.82.4:4802 , piece_id=WIO6KTPX43VHXO64HRMO5KGJ7FD2P6DMSAUKEYV5P7R5NH4XRQHQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487637044 (60h42m34.67793563s) [noise=true , address=104.36.82.15:4258 , piece_id=2YWOHD3RSNBD7K24ZD6JJJOERI4MLBMDPRITYCUFKQHRDTH4MYFA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487900358 (60h42m34.678198945s) [noise=true , address=45.91.116.13:4154 , piece_id=KSR45P7YV4HEABWB5CVQM4NFL7MUNKJTGV4JS3CJ6SRZQYXZ2DHA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.487931250 (60h42m34.678229831s) [noise=true , address=77.90.31.16:4733 , piece_id=NSGBPILBI3UYR2L57T2Q2W3MUID7J7M6QKPMKZJ7HYUCSEO77WBA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.488296846 (60h42m34.678595427s) [noise=true , address=45.91.116.5:4298 , piece_id=NLMFP5NK6F5ORI754VHNPIDLTKL5BTPY7SES6BFXG4KIOAUZQH6Q, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.488596786 (60h42m34.678895365s) [noise=true , address=45.91.116.7:4011 , piece_id=4BDNHSORHLCE54CCFG6TLY2XNDPHNZ2F6ZIHVS73NVRP553Y3JYQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489061071 (60h42m34.679359654s) [noise=true , address=45.91.116.6:4973 , piece_id=N4HN73VB6DHJ57ERGBADR5OYNGMENYGWTHGXQBZSZSJM5T47PNFQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489376608 (60h42m34.679675187s) [noise=true , address=45.91.116.4:4939 , piece_id=FB6QKW5GMFL2NMRAAYU3A5EBVKE7WUMX6FQAS65AVL726HSL3PMA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489708319 (60h42m34.680006908s) [noise=true , address=45.91.116.6:4975 , piece_id=T24TA7MTX3KUB3DRDJYYPOUEAWIMNK2KHBZPZVZFEAQSQQSUHDAA, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.489979774 (60h42m34.680278366s) [noise=true , address=104.36.82.9:4007 , piece_id=ZBKT7APSNONYS2ERXUROUVGCXVBW27VVFLSJK4TTR4I725LKOQ4Q, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.490367371 (60h42m34.680665962s) [noise=true , address=45.91.116.12:4225 , piece_id=UGWVN4GYV67MHU4ATDO53763FIH7ZPANW7MCEJLE4CZ75AEZIK7Q, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.490713165 (60h42m34.681011749s) [noise=true , address=77.90.31.15:4699 , piece_id=6VYS4G2C6FWVUYAM2FV74X4NCSPEHPK2R34EJRVUM7PLSSYPDGOQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.491120809 (60h42m34.681419388s) [noise=true , address=77.90.31.12:4589 , piece_id=BGZOTIBEZODJU4Z3WACJBR22TVK4JTVHBRBHULAM42UXVMTXNKFQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.491486611 (60h42m34.681785193s) [noise=true , address=45.91.116.14:4189 , piece_id=2N3RHWIGQISGX6MANQ37RT4GBYZOXWOKPVTLQJDNURO5DM2JV3JQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:03.491815645 (60h42m34.682114234s) [noise=true , address=104.36.82.6:4873 , piece_id=NOMBRMCMTZGRFGDLFHCEIIIDNU3E3VGIUVRH7AUB2YPZI3NDTNKQ, seg_pos={0 0}, upload=58192]: Uploading piece... Jul 27 00:18:04.100400461 (60h42m35.29069905s) [upload=58192]: Next returned nil segment Jul 27 00:18:04.100425241 (60h42m35.290723832s) [upload=58192]: Waiting for error group managing segments... Jul 27 00:18:04.237866049 (60h42m35.428164619s) [noise=true , address=77.90.31.15:4699 , piece_id=6VYS4G2C6FWVUYAM2FV74X4NCSPEHPK2R34EJRVUM7PLSSYPDGOQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.243114679 (60h42m35.433413244s) [noise=true , address=45.91.116.4:4939 , piece_id=FB6QKW5GMFL2NMRAAYU3A5EBVKE7WUMX6FQAS65AVL726HSL3PMA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.246965506 (60h42m35.43726409s) [noise=true , address=45.91.116.7:4011 , piece_id=4BDNHSORHLCE54CCFG6TLY2XNDPHNZ2F6ZIHVS73NVRP553Y3JYQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.247104772 (60h42m35.437403355s) [noise=true , address=45.91.116.13:4154 , piece_id=KSR45P7YV4HEABWB5CVQM4NFL7MUNKJTGV4JS3CJ6SRZQYXZ2DHA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.248545042 (60h42m35.43884361s) [noise=true , address=45.91.116.6:4973 , piece_id=N4HN73VB6DHJ57ERGBADR5OYNGMENYGWTHGXQBZSZSJM5T47PNFQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.248681535 (60h42m35.438980103s) [noise=true , address=45.91.116.6:4975 , piece_id=T24TA7MTX3KUB3DRDJYYPOUEAWIMNK2KHBZPZVZFEAQSQQSUHDAA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.254634547 (60h42m35.444933129s) [noise=true , address=45.91.116.5:4298 , piece_id=NLMFP5NK6F5ORI754VHNPIDLTKL5BTPY7SES6BFXG4KIOAUZQH6Q, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.256124428 (60h42m35.446422998s) [noise=true , address=77.90.31.16:4733 , piece_id=NSGBPILBI3UYR2L57T2Q2W3MUID7J7M6QKPMKZJ7HYUCSEO77WBA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.257560649 (60h42m35.447859235s) [noise=true , address=45.91.116.12:4225 , piece_id=UGWVN4GYV67MHU4ATDO53763FIH7ZPANW7MCEJLE4CZ75AEZIK7Q, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:04.528204477 (60h42m35.718503043s) [noise=true , address=45.91.116.14:4189 , piece_id=2N3RHWIGQISGX6MANQ37RT4GBYZOXWOKPVTLQJDNURO5DM2JV3JQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:05.628152792 (60h42m36.818451377s) [noise=true , address=77.90.31.13:4628 , piece_id=OCTUFUZKUA6SDJN4QVALXPCGYULF75XL73M2BPGXEJTSJSJHLUNQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:05.777962181 (60h42m36.968260754s) [noise=true , address=77.90.31.12:4589 , piece_id=BGZOTIBEZODJU4Z3WACJBR22TVK4JTVHBRBHULAM42UXVMTXNKFQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: <nil> Jul 27 00:18:33.101710794 (60h43m4.292009373s) [noise=true , address=104.36.82.4:4802 , piece_id=WIO6KTPX43VHXO64HRMO5KGJ7FD2P6DMSAUKEYV5P7R5NH4XRQHQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 139.59.144.22:41636->104.36.82.4:4802: use of closed network connection; write tcp 139.59.144.22:41636->104.36.82.4:4802: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41636->104.36.82.4:4802: use of closed network connection Jul 27 00:18:33.101810707 (60h43m4.292109286s) [noise=true , address=104.36.82.9:4007 , piece_id=ZBKT7APSNONYS2ERXUROUVGCXVBW27VVFLSJK4TTR4I725LKOQ4Q, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: write tcp 139.59.144.22:46944->104.36.82.9:4007: use of closed network connection; write tcp 139.59.144.22:46944->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: context deadline exceeded Jul 27 00:18:33.101886971 (60h43m4.292185551s) [noise=true , address=104.36.82.15:4258 , piece_id=2YWOHD3RSNBD7K24ZD6JJJOERI4MLBMDPRITYCUFKQHRDTH4MYFA, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: write tcp 139.59.144.22:54986->104.36.82.15:4258: use of closed network connection; write tcp 139.59.144.22:54986->104.36.82.15:4258: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:54986->104.36.82.15:4258: use of closed network connection Jul 27 00:18:33.101992511 (60h43m4.292291099s) [noise=true , address=104.36.82.6:4873 , piece_id=NOMBRMCMTZGRFGDLFHCEIIIDNU3E3VGIUVRH7AUB2YPZI3NDTNKQ, seg_pos={0 0}, upload=58192]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:104.36.82.6:4873): protocol: write tcp 139.59.144.22:34762->104.36.82.6:4873: use of closed network connection; write tcp 139.59.144.22:34762->104.36.82.6:4873: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:34762->104.36.82.6:4873: use of closed network connection Jul 27 00:18:33.102056717 (60h43m4.292355309s) [seg_pos={0 0}, upload=58192]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 27 00:18:33.203833358 (60h43m4.39413193s) [upload=58192]: Done uploading segments... Jul 27 00:18:33.203860208 (60h43m4.394158787s) [upload=58192]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.104487187s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.....X..XXX....XXX...X....X.......XX...X....X...XX....X.....] duration: 4.045435594s FAIL (2025-07-27 00:18:28 UTC) orbiter warning: context deadline exceeded Duration: 4.045475641s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [X..X.....XX.X.X......X..X...X.X.X..X..XXX....XX......X..X..X] duration: 1m0.002032223s FAIL (2025-07-27 00:18:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 29565816s) [upload=58173]: Upload finished. err: <nil> Jul 27 00:17:26.203446547 (60h41m57.393745146s) [upload=58185]: Starting upload Jul 27 00:17:26.203572895 (60h41m57.393871492s) [upload=58185]: Done starting upload Jul 27 00:17:26.203732135 (60h41m57.394030706s) [upload=58185]: Uploading segments... Jul 27 00:17:26.204105900 (60h41m57.394404495s) [upload=58185]: Got next segment. Inline: false Jul 27 00:17:26.204158466 (60h41m57.394457036s) [upload=58185]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:17:26.204164645 (60h41m57.394463224s) [upload=58185]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:17:26.404507738 (60h41m57.594806317s) [seg_pos={0 0}, upload=58185]: Begin upload segment... Jul 27 00:17:26.404566965 (60h41m57.594865531s) [seg_pos={0 0}, upload=58185]: Done begin upload segment. Jul 27 00:17:26.404606437 (60h41m57.594905025s) [noise=true , address=104.36.82.51:4060 , piece_id=CQFBAG6VREGZGSAJPZXFYYKPTWPTPL6UZBFZ3PG3PJTZAMSJTT7A, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.404622148 (60h41m57.594920727s) [noise=true , address=77.90.31.18:4840 , piece_id=TAJ4NYCL2AFHGWX6WPIGBA4BQQKIB6U6YVO7WF6BTWMMCFK3ASLQ, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.405075332 (60h41m57.595373913s) [noise=true , address=79.127.201.215:4347 , piece_id=2FWBGVVH7AUYASLJTFTTEGZ45T2KOPZPXAN75K7JMBDESIN4UHWA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.405188220 (60h41m57.595486806s) [noise=true , address=77.90.31.15:4730 , piece_id=ADZYVLJWSTCFT5C3PDG4GYGNFK4CAVPRCTLT67VWUOURJHUMD7PQ, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.405470995 (60h41m57.595769568s) [noise=true , address=45.91.116.14:4194 , piece_id=JQYBU7JTZTQC254ETMZ4BCCSLQ34KKHJLZ3GSZ3SE4VGSWOYXG4Q, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.405568342 (60h41m57.595866932s) [noise=true , address=104.36.82.47:4542 , piece_id=BDXVQAHLXGYHZDOPQIASFIEX7V7FKP7TSIVOPXABPM3CCJOVUZSQ, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.406012522 (60h41m57.596311102s) [noise=true , address=79.127.205.233:4656 , piece_id=GSX6F2EDKVFLI44FDPQRQ54MQ5MQ37VKCJJXWSARGIEMC3J4I73Q, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.406387027 (60h41m57.596685606s) [noise=true , address=79.127.205.245:4246 , piece_id=BIYAYTKBPAVPGTXLUC76R2BLZ6SGA2ICMS5R2NUHDXVZF4ZAR4WA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.406503895 (60h41m57.59680248s) [noise=true , address=104.36.82.46:4495 , piece_id=PHFBE6WBSETH7JOGMKMWTJQXJALERUJ5YI2E22YHRVALX4LAYPLA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.406759129 (60h41m57.597057709s) [noise=true , address=104.36.82.56:4241 , piece_id=5JEJY6JVH57IC65PGB7PEPI5UFRIIZARNMFP5BPTBWROWFLQODBQ, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.407213454 (60h41m57.597512036s) [noise=true , address=45.91.116.9:4277 , piece_id=6GP763MCP3JXJOPQNOF6JG4L7ELUNAQLG55Q7LFHE5PCXSLCUVHA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.407382572 (60h41m57.597681161s) [noise=true , address=79.127.220.101:4041 , piece_id=DQ75OP2B24TN5YWKJEK72LBWWQHA27JZZBFG4AH5RRE3JETXLJMA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.407591055 (60h41m57.597889637s) [noise=true , address=64.49.13.138:4483 , piece_id=DJ4KS3IHYNWOTSM5D6R3N4C7E7B2DZWLQTWJIBGO7FBRUJKMY7HQ, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.407932118 (60h41m57.5982307s) [noise=true , address=104.36.82.52:4110 , piece_id=SVHJRG3OFL3Y2VKHWWEOCWY6YSOIHFAMB45JCIC5MY7VZGZYWRGA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.407985304 (60h41m57.598283874s) [noise=true , address=79.127.219.38:4009 , piece_id=CLTZK7HZBEA73XBN4SRXATESS2EZJTEAZVLYCUG7AYUW54FYPYCA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.408299094 (60h41m57.598597676s) [noise=true , address=104.36.82.53:4149 , piece_id=RJN4RGNRXLAMBZ7M3MP2V7EL4WBMY4PIJOGMRAQ7ZC5AYBEEPHOA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.408648200 (60h41m57.598946782s) [noise=true , address=64.49.13.140:4514 , piece_id=IRFSFAELRSLQZ4WRYHJAGGZPYP32JZS76BOJI6VIDIMXJW57D5PA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.409047060 (60h41m57.599345645s) [noise=true , address=77.90.31.11:4584 , piece_id=4RRJ5KYHBWYXGVKFPW3B34SZLGKUDTRAOUUQAEZYRFAY6C7FSZYA, seg_pos={0 0}, upload=58185]: Uploading piece... Jul 27 00:17:26.914676825 (60h41m58.104975407s) [noise=true , address=79.127.220.101:4041 , piece_id=DQ75OP2B24TN5YWKJEK72LBWWQHA27JZZBFG4AH5RRE3JETXLJMA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: ecclient: upload failed (node:123G35C5bhcQpSSGRtKGo78ZfxNj6bZatY3kCrdEu1kMyjV5Yiw, address:79.127.220.101:4041): protocol: storage node overloaded, request limit: 2000 Jul 27 00:17:26.980386347 (60h41m58.170684935s) [upload=58185]: Next returned nil segment Jul 27 00:17:26.980414902 (60h41m58.170713475s) [upload=58185]: Waiting for error group managing segments... Jul 27 00:17:27.153330233 (60h41m58.3436288s) [noise=true , address=77.90.31.18:4840 , piece_id=TAJ4NYCL2AFHGWX6WPIGBA4BQQKIB6U6YVO7WF6BTWMMCFK3ASLQ, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:27.299913150 (60h41m58.490211718s) [noise=true , address=79.127.219.38:4009 , piece_id=CLTZK7HZBEA73XBN4SRXATESS2EZJTEAZVLYCUG7AYUW54FYPYCA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:27.408077805 (60h41m58.598376375s) [noise=true , address=77.90.31.11:4584 , piece_id=4RRJ5KYHBWYXGVKFPW3B34SZLGKUDTRAOUUQAEZYRFAY6C7FSZYA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:27.606695977 (60h41m58.796994559s) [noise=true , address=64.49.13.140:4514 , piece_id=IRFSFAELRSLQZ4WRYHJAGGZPYP32JZS76BOJI6VIDIMXJW57D5PA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: ecclient: upload failed (node:1WsbDX2Zy8FYnEQtUsYFHSeDHj1DkkGyWBPaxzmReWFzaTejHn, address:64.49.13.140:4514): protocol: storage node overloaded, request limit: 2000 Jul 27 00:17:27.645821869 (60h41m58.836120453s) [noise=true , address=79.127.205.245:4246 , piece_id=BIYAYTKBPAVPGTXLUC76R2BLZ6SGA2ICMS5R2NUHDXVZF4ZAR4WA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:27.725957801 (60h41m58.916256374s) [noise=true , address=79.127.205.233:4656 , piece_id=GSX6F2EDKVFLI44FDPQRQ54MQ5MQ37VKCJJXWSARGIEMC3J4I73Q, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:27.774541086 (60h41m58.964839666s) [noise=true , address=77.90.31.15:4730 , piece_id=ADZYVLJWSTCFT5C3PDG4GYGNFK4CAVPRCTLT67VWUOURJHUMD7PQ, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:28.383980259 (60h41m59.574278846s) [noise=true , address=64.49.13.138:4483 , piece_id=DJ4KS3IHYNWOTSM5D6R3N4C7E7B2DZWLQTWJIBGO7FBRUJKMY7HQ, seg_pos={0 0}, upload=58185]: Done uploading piece. err: ecclient: upload failed (node:12Bq23DceaRvomaGDHoZr3sYzSHHJEQr2wE4LzSEYk9rUm1DhLa, address:64.49.13.138:4483): protocol: storage node overloaded, request limit: 2000 Jul 27 00:17:32.297310152 (60h42m3.487608721s) [noise=true , address=45.91.116.9:4277 , piece_id=6GP763MCP3JXJOPQNOF6JG4L7ELUNAQLG55Q7LFHE5PCXSLCUVHA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:34.739212370 (60h42m5.92951094s) [noise=true , address=104.36.82.53:4149 , piece_id=RJN4RGNRXLAMBZ7M3MP2V7EL4WBMY4PIJOGMRAQ7ZC5AYBEEPHOA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:42.946176737 (60h42m14.136475304s) [noise=true , address=45.91.116.14:4194 , piece_id=JQYBU7JTZTQC254ETMZ4BCCSLQ34KKHJLZ3GSZ3SE4VGSWOYXG4Q, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:45.923491251 (60h42m17.113789825s) [noise=true , address=104.36.82.52:4110 , piece_id=SVHJRG3OFL3Y2VKHWWEOCWY6YSOIHFAMB45JCIC5MY7VZGZYWRGA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:17:59.649119760 (60h42m30.839418324s) [noise=true , address=79.127.201.215:4347 , piece_id=2FWBGVVH7AUYASLJTFTTEGZ45T2KOPZPXAN75K7JMBDESIN4UHWA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:18:01.307512002 (60h42m32.497810642s) [noise=true , address=104.36.82.56:4241 , piece_id=5JEJY6JVH57IC65PGB7PEPI5UFRIIZARNMFP5BPTBWROWFLQODBQ, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:18:15.520050994 (60h42m46.71034956s) [noise=true , address=104.36.82.46:4495 , piece_id=PHFBE6WBSETH7JOGMKMWTJQXJALERUJ5YI2E22YHRVALX4LAYPLA, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:18:25.125782572 (60h42m56.316081152s) [noise=true , address=104.36.82.47:4542 , piece_id=BDXVQAHLXGYHZDOPQIASFIEX7V7FKP7TSIVOPXABPM3CCJOVUZSQ, seg_pos={0 0}, upload=58185]: Done uploading piece. err: <nil> Jul 27 00:18:25.125822747 (60h42m56.316121336s) [seg_pos={0 0}, upload=58185]: Segment reached optimal threshold of 14 pieces. Jul 27 00:18:25.126003883 (60h42m56.316302464s) [noise=true , address=104.36.82.51:4060 , piece_id=CQFBAG6VREGZGSAJPZXFYYKPTWPTPL6UZBFZ3PG3PJTZAMSJTT7A, seg_pos={0 0}, upload=58185]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19VJKnTAges9YKYdUVoz3tb3sFU1bawxpaJ2rsBd2wmLo1zK4X): protocol: expected piece hash; context canceled; EOF Jul 27 00:18:25.126029608 (60h42m56.316328187s) [seg_pos={0 0}, upload=58185]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 27 00:18:25.126128822 (60h42m56.316427403s) [upload=58185]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:18:25.126134573 (60h42m56.316433155s) [upload=58185]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:18:25.272947461 (60h42m56.463246031s) [upload=58185]: Done uploading segments... Jul 27 00:18:25.272980842 (60h42m56.463279411s) [upload=58185]: Upload finished. err: <nil> Duration: 1m0.002062236s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [.......XX.X....X......X..X............X.X..X.X..............] duration: 24.256417993s FAIL (2025-07-27 00:18:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 24.256434843s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 37 out of 60 is failed (threshold=3) [XXXXXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X.X..] duration: 3.712096253s FAIL (2025-07-27 00:18:01 UTC) orbiter warning: uplink: context deadline exceeded Duration: 3.71213399s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 10 out of 60 is failed (threshold=3) [........XX.X....X......X..X............X.X..X.X.............] duration: 56.005378182s FAIL (2025-07-27 00:17:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 56.005396901s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.......................X.X......] duration: 963.659051ms FAIL (2025-07-27 00:17:29 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 963.692182ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [......X..XXX....XXX...X....X.......XX...X....X...XX....X....] duration: 3.8336033s FAIL (2025-07-27 00:17:28 UTC) orbiter warning: context deadline exceeded Duration: 3.833764109s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [XX..X.....XX.X.X......X..X...X.X.X..X..XXX....XX......X..X..] duration: 55.732033467s FAIL (2025-07-27 00:17:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:14:42.538877658 (60h39m13.729176236s) [upload=58146]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:14:42.739636704 (60h39m13.929935299s) [seg_pos={0 0}, upload=58146]: Begin upload segment... Jul 27 00:14:42.739701279 (60h39m13.929999858s) [seg_pos={0 0}, upload=58146]: Done begin upload segment. Jul 27 00:14:42.739741629 (60h39m13.930040209s) [noise=true , address=77.90.31.10:4528 , piece_id=JVJHEJDNGP6GZL2XOW22JJ5NRWY3GMAU4UJ6O6HPFYLTVOBNIYUQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.739750833 (60h39m13.930049415s) [noise=true , address=104.36.82.41:4772 , piece_id=JYQ6P4EYCLWJJV7HZUU6DAFYMBQJPOEJ5NEFAIJVWER3CLWWYKEA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740135209 (60h39m13.930433808s) [noise=true , address=104.36.82.5:4854 , piece_id=6OJS4SYNSIH2VHL7TFQRLTXVGRBFVZA3OIPEA7Y2UQDI7IIUW7QA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740188883 (60h39m13.930487468s) [noise=true , address=79.127.201.215:4356 , piece_id=MWRVNKONLFS574ATTF4ZM2MJGXVPVVJ4UIS6ETYDEQLT22CLIAFA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740449586 (60h39m13.930748172s) [noise=true , address=104.36.82.56:4241 , piece_id=C2TWLPSOM2OWCLJVCE2BIR2RAFFHUX2C3BYDHH4CCSGSXEP7IHDA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740524021 (60h39m13.930822603s) [noise=true , address=77.90.31.13:4644 , piece_id=D3UGBM7OM5OHTDBCFVNVIJGKCUWEM2GRAOL3XKYPGNTJNYO4L23A, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740884918 (60h39m13.931183507s) [noise=true , address=79.127.219.48:4074 , piece_id=2CL3CL3TEXWGBNFJO5X3X32HDDOVIFCBKDDU3APHYTDPE27BRNAQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741260115 (60h39m13.931558709s) [noise=true , address=64.49.13.140:4508 , piece_id=J2KV33QQHCUAVLBSABR4XVY3VXU6ZDS52XCTAEBRCVH2LY5EBCYA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741615447 (60h39m13.931914273s) [noise=true , address=79.127.205.246:4284 , piece_id=VLZRH5XZFEWW4W6LWCZ6ZG3JAQYNBW6CRI6RQT4J376L44KOAFHQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741990455 (60h39m13.932289044s) [noise=true , address=104.36.82.46:4489 , piece_id=T3UH4QBINTILCG4CGXUFS6LNPALQ5IJWLFBFKTDTBEKV7QBNF53Q, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742045465 (60h39m13.932344055s) [noise=true , address=104.36.82.54:4169 , piece_id=S452MUJKZV7RFCYWYVHCV6B5KMVCGUYZ66DIOYHTKNA6VLE7BNSA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742403647 (60h39m13.932702234s) [noise=true , address=77.90.31.11:4579 , piece_id=I5M4EFRTPP6YCGM3RDHCUYYARAR5U2AJNHJQTOFQFP3I5UL3FT3Q, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742852820 (60h39m13.933151405s) [noise=true , address=104.36.82.8:4999 , piece_id=SKWZANHJTKKQPI4DXRAX7Q4W3J5LUCXSXFZWBLPCCHB7NJZ2DNTA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743182186 (60h39m13.933480771s) [noise=true , address=79.127.205.245:4247 , piece_id=QWEM6REHHNRH6ZRKH42ZFNCNSWOTGGPZT5U4O24ANT6FQXSG4MQA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743537911 (60h39m13.933836494s) [noise=true , address=79.127.219.38:4017 , piece_id=AEMM6SRH2QHLOE4FH2EYCSG3PG7KFX7KQYGXJ6XAMIVQPYGT3DMQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743894189 (60h39m13.93419278s) [noise=true , address=104.36.82.47:4547 , piece_id=2K2PFVND2GJY6JQ4WPRXLHY6ATAFLUGZUQD5TXGEFWHZPTHSGZCA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.744216490 (60h39m13.934515072s) [noise=true , address=77.90.31.21:4909 , piece_id=GYN722GGZX7SFTJFNE7JFFQCF5JYDAJMNSUUDZSYT4JDUBCY6ETQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.744506947 (60h39m13.934805518s) [noise=true , address=77.90.31.16:4736 , piece_id=VFM2T2ZVOMXQ5WWHGTACKXZONVP4IKIZA2RCKVZVMQYAZNSUFFJQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:43.313073027 (60h39m14.503371612s) [upload=58146]: Next returned nil segment Jul 27 00:14:43.313095000 (60h39m14.503393588s) [upload=58146]: Waiting for error group managing segments... Jul 27 00:14:43.495135149 (60h39m14.68543372s) [noise=true , address=77.90.31.16:4736 , piece_id=VFM2T2ZVOMXQ5WWHGTACKXZONVP4IKIZA2RCKVZVMQYAZNSUFFJQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.518601307 (60h39m14.708899891s) [noise=true , address=77.90.31.11:4579 , piece_id=I5M4EFRTPP6YCGM3RDHCUYYARAR5U2AJNHJQTOFQFP3I5UL3FT3Q, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.533843430 (60h39m14.724142005s) [noise=true , address=79.127.219.38:4017 , piece_id=AEMM6SRH2QHLOE4FH2EYCSG3PG7KFX7KQYGXJ6XAMIVQPYGT3DMQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.544355195 (60h39m14.734653762s) [noise=true , address=79.127.219.48:4074 , piece_id=2CL3CL3TEXWGBNFJO5X3X32HDDOVIFCBKDDU3APHYTDPE27BRNAQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.987358494 (60h39m15.177657073s) [noise=true , address=79.127.205.246:4284 , piece_id=VLZRH5XZFEWW4W6LWCZ6ZG3JAQYNBW6CRI6RQT4J376L44KOAFHQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:44.088812791 (60h39m15.27911136s) [noise=true , address=79.127.205.245:4247 , piece_id=QWEM6REHHNRH6ZRKH42ZFNCNSWOTGGPZT5U4O24ANT6FQXSG4MQA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:46.408185626 (60h39m17.598484211s) [noise=true , address=77.90.31.13:4644 , piece_id=D3UGBM7OM5OHTDBCFVNVIJGKCUWEM2GRAOL3XKYPGNTJNYO4L23A, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:50.957512682 (60h39m22.147811262s) [noise=true , address=64.49.13.140:4508 , piece_id=J2KV33QQHCUAVLBSABR4XVY3VXU6ZDS52XCTAEBRCVH2LY5EBCYA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:00.579829869 (60h39m31.770128446s) [noise=true , address=104.36.82.56:4241 , piece_id=C2TWLPSOM2OWCLJVCE2BIR2RAFFHUX2C3BYDHH4CCSGSXEP7IHDA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:06.746898478 (60h39m37.937197061s) [noise=true , address=104.36.82.54:4169 , piece_id=S452MUJKZV7RFCYWYVHCV6B5KMVCGUYZ66DIOYHTKNA6VLE7BNSA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:18.131452986 (60h39m49.321751559s) [noise=true , address=79.127.201.215:4356 , piece_id=MWRVNKONLFS574ATTF4ZM2MJGXVPVVJ4UIS6ETYDEQLT22CLIAFA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:25.447346116 (60h39m56.637644705s) [noise=true , address=104.36.82.47:4547 , piece_id=2K2PFVND2GJY6JQ4WPRXLHY6ATAFLUGZUQD5TXGEFWHZPTHSGZCA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:42.261804835 (60h40m13.452103415s) [noise=true , address=104.36.82.5:4854 , piece_id=6OJS4SYNSIH2VHL7TFQRLTXVGRBFVZA3OIPEA7Y2UQDI7IIUW7QA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy, address:104.36.82.5:4854): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.261997725 (60h40m13.452296305s) [noise=true , address=104.36.82.41:4772 , piece_id=JYQ6P4EYCLWJJV7HZUU6DAFYMBQJPOEJ5NEFAIJVWER3CLWWYKEA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:1zKCizjMAKNzq6CvZaVBJmjox6abJzywmtbjCUS1dWcDV2qSip, address:104.36.82.41:4772): protocol: expected piece hash; write tcp 139.59.144.22:60272->104.36.82.41:4772: use of closed network connection; write tcp 139.59.144.22:60272->104.36.82.41:4772: use of closed network connection; EOF Jul 27 00:15:42.262040696 (60h40m13.452339275s) [noise=true , address=77.90.31.10:4528 , piece_id=JVJHEJDNGP6GZL2XOW22JJ5NRWY3GMAU4UJ6O6HPFYLTVOBNIYUQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:15fkvAWei1E5Tn2BYkCavyZ8se6XwPJcmpr7aEsv8sbMtk1n3Q, address:77.90.31.10:4528): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.262053263 (60h40m13.452351835s) [noise=true , address=104.36.82.8:4999 , piece_id=SKWZANHJTKKQPI4DXRAX7Q4W3J5LUCXSXFZWBLPCCHB7NJZ2DNTA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:12Y993kGnB4JusXCcZ4KGyGpRBwoHksVBPzK77ePMw9G7djQEyY, address:104.36.82.8:4999): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.262101355 (60h40m13.452399935s) [noise=true , address=104.36.82.46:4489 , piece_id=T3UH4QBINTILCG4CGXUFS6LNPALQ5IJWLFBFKTDTBEKV7QBNF53Q, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:12YbmXQeh182PYH1AHxvGcw3zW3uiuDsbYPomH3HAPNtjbJSpPw, address:104.36.82.46:4489): protocol: write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection; write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection Jul 27 00:15:42.262242633 (60h40m13.452541213s) [noise=true , address=77.90.31.21:4909 , piece_id=GYN722GGZX7SFTJFNE7JFFQCF5JYDAJMNSUUDZSYT4JDUBCY6ETQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:121RT7tP6czMFAiZ3DjzSSsYgPt3xZXVbXVpkcYxaZYscNiMjL6, address:77.90.31.21:4909): protocol: read tcp 139.59.144.22:48056->77.90.31.21:4909: use of closed network connection; context deadline exceeded Jul 27 00:15:42.262288698 (60h40m13.452587277s) [seg_pos={0 0}, upload=58146]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 27 00:15:42.370689999 (60h40m13.560988579s) [upload=58146]: Done uploading segments... Jul 27 00:15:42.370716421 (60h40m13.561015s) [upload=58146]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 55.732060939s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 38 out of 60 is failed (threshold=3) [XXXXXXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X.X.] duration: 2.818453821s FAIL (2025-07-27 00:17:18 UTC) orbiter warning: uplink: context deadline exceeded Duration: 2.818488173s core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [X......X..XXX....XXX...X....X.......XX...X....X...XX....X...] duration: 2.695591588s FAIL (2025-07-27 00:17:07 UTC) orbiter warning: context deadline exceeded Duration: 2.695621168s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 36 out of 60 is failed (threshold=3) [XX.XXXXXXXXXXXXXXX..X.X.X.XX.XXXXX..XX.XX..XXXX.X...........] duration: 3.75491859s FAIL (2025-07-27 00:16:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== d: op Jul 27 00:09:10.565262576 (60h33m41.755561156s) [seg_pos={0 0}, upload=58055]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 27 00:09:10.565365113 (60h33m41.755663693s) [upload=58055]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:09:10.565370840 (60h33m41.75566942s) [upload=58055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:09:10.718568801 (60h33m41.908867373s) [upload=58055]: Done uploading segments... Jul 27 00:09:10.718606181 (60h33m41.90890476s) [upload=58055]: Upload finished. err: <nil> Jul 27 00:09:16.562449298 (60h33m47.752747867s) [upload=58061]: Starting upload Jul 27 00:09:16.562551137 (60h33m47.752849721s) [upload=58061]: Done starting upload Jul 27 00:09:16.562688075 (60h33m47.752986646s) [upload=58061]: Uploading segments... Jul 27 00:09:16.562953051 (60h33m47.753251622s) [upload=58061]: Got next segment. Inline: false Jul 27 00:09:16.563018592 (60h33m47.753317172s) [upload=58061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:09:16.563024426 (60h33m47.753323004s) [upload=58061]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:09:16.674481109 (60h33m47.864779681s) [seg_pos={0 0}, upload=58061]: Begin upload segment... Jul 27 00:09:16.674530332 (60h33m47.864828901s) [seg_pos={0 0}, upload=58061]: Done begin upload segment. Jul 27 00:09:16.674569232 (60h33m47.864867821s) [noise=true , address=104.36.82.10:4078 , piece_id=DWLO4ER732CXF4UNSFBN4SE6BWSHSZBNSGFJ3KY2GP6NPOKAVJWA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.674588419 (60h33m47.864887008s) [noise=true , address=104.36.82.4:4801 , piece_id=CL2IRP22RCTBH3VYQTN7VJ4XPZNBSEJF2O3US4EMPPGWYZTL2MTA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.675107134 (60h33m47.865405733s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMJGODVGCXOJ4TJFPBJKSIEHHQJG2MP7Z7O7B7XCSP7AS4NKAB5A, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.675946131 (60h33m47.866244721s) [noise=true , address=104.36.82.8:4972 , piece_id=ER6QEA7NDGF7WKGZQM6KOXDQ72GRJUCOPTB4X4SFEF2VJ74WKTHA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.676410846 (60h33m47.866709436s) [noise=true , address=45.91.116.11:4118 , piece_id=LKLI6NIPKUYOYQ24N7S7IQ3A43SOM4D5C2AT7LSG4QD463UT3LRA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.677078109 (60h33m47.867376698s) [noise=true , address=77.90.31.17:4771 , piece_id=FLPVNCITXTBBOQRGCKO3B2VQSYAOCPER3CN7DM6DQHM3OW3BXV2Q, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.677277156 (60h33m47.867575737s) [noise=true , address=45.91.116.12:4227 , piece_id=FQOPTSR7UI65EF3UXDTULVD7ZVFWRDRWUC6OSUXSI6VIKEZNXDDQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.677819926 (60h33m47.868118506s) [noise=true , address=104.36.82.12:4149 , piece_id=BXUFNBPQ3MP6EUXQ4FJU45EGQ47J7FPK6ALELVQ2ILHOZLLUFL4Q, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.678383329 (60h33m47.868681921s) [noise=true , address=45.91.116.6:4975 , piece_id=OVDDE74WATWIUHTVB3UTX4P7LD3JO3R2KRDWZW7YU4LCLUKUZQPQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.679073127 (60h33m47.869371716s) [noise=true , address=45.91.116.7:4011 , piece_id=QXNMHUAXWDEBPGP2JHQA6UKWAYN5PKTVRXOLMZM3PBEARYT4VLDA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.679542711 (60h33m47.869841303s) [noise=true , address=45.91.116.10:4082 , piece_id=VW45M6QMGAPSJ7WOTPL5JUDNWGWPNX2TYVLCJAWOTLO3J3DXYLAQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.680001782 (60h33m47.870300373s) [noise=true , address=77.90.31.13:4626 , piece_id=E6D4YLFW2OPSSH3B6GLQIFZPB7RFS2DKHEBHDTMKJOBINVISUMCQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.680556884 (60h33m47.870855472s) [noise=true , address=45.91.116.13:4154 , piece_id=IYW5WRAIUVWDSY6HDN2ZHJM5NRXNKGOVVIH7SCUKIXPOYVMS56BQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.681059039 (60h33m47.871357622s) [noise=true , address=104.36.82.9:4006 , piece_id=NVAH7SXF2G4FJ6FI7XRCW4RE5ZQEVPV7NCLQYBE6UKWDPZPPO4EA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.681557482 (60h33m47.871856071s) [noise=true , address=104.36.82.14:4223 , piece_id=YW7GUOEGIXNZEN33TDQX7IDVJGO4ZGU7GWNKVLIOQ54HBVAYSIDQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.682063049 (60h33m47.872361638s) [noise=true , address=45.91.116.14:4190 , piece_id=X3SDGPNCQPWJMJV7KBRYQLC5LEGX6CDAHWLWHRRGOVG5WVTW3C3A, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:17.316057568 (60h33m48.506356156s) [upload=58061]: Next returned nil segment Jul 27 00:09:17.316080757 (60h33m48.506379348s) [upload=58061]: Waiting for error group managing segments... Jul 27 00:09:17.422557859 (60h33m48.61285643s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMJGODVGCXOJ4TJFPBJKSIEHHQJG2MP7Z7O7B7XCSP7AS4NKAB5A, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.424121032 (60h33m48.614419601s) [noise=true , address=45.91.116.6:4975 , piece_id=OVDDE74WATWIUHTVB3UTX4P7LD3JO3R2KRDWZW7YU4LCLUKUZQPQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.445556516 (60h33m48.635855187s) [noise=true , address=45.91.116.7:4011 , piece_id=QXNMHUAXWDEBPGP2JHQA6UKWAYN5PKTVRXOLMZM3PBEARYT4VLDA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.447312205 (60h33m48.637610778s) [noise=true , address=45.91.116.13:4154 , piece_id=IYW5WRAIUVWDSY6HDN2ZHJM5NRXNKGOVVIH7SCUKIXPOYVMS56BQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.455506767 (60h33m48.645805346s) [noise=true , address=45.91.116.10:4082 , piece_id=VW45M6QMGAPSJ7WOTPL5JUDNWGWPNX2TYVLCJAWOTLO3J3DXYLAQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.460800503 (60h33m48.651099087s) [noise=true , address=45.91.116.11:4118 , piece_id=LKLI6NIPKUYOYQ24N7S7IQ3A43SOM4D5C2AT7LSG4QD463UT3LRA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.464762217 (60h33m48.6550608s) [noise=true , address=77.90.31.17:4771 , piece_id=FLPVNCITXTBBOQRGCKO3B2VQSYAOCPER3CN7DM6DQHM3OW3BXV2Q, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.523143661 (60h33m48.713442242s) [noise=true , address=45.91.116.12:4227 , piece_id=FQOPTSR7UI65EF3UXDTULVD7ZVFWRDRWUC6OSUXSI6VIKEZNXDDQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.581308187 (60h33m48.771606753s) [noise=true , address=45.91.116.14:4190 , piece_id=X3SDGPNCQPWJMJV7KBRYQLC5LEGX6CDAHWLWHRRGOVG5WVTW3C3A, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:18.942817938 (60h33m50.133116526s) [noise=true , address=77.90.31.13:4626 , piece_id=E6D4YLFW2OPSSH3B6GLQIFZPB7RFS2DKHEBHDTMKJOBINVISUMCQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:26.999729278 (60h33m58.190027866s) [noise=true , address=104.36.82.4:4801 , piece_id=CL2IRP22RCTBH3VYQTN7VJ4XPZNBSEJF2O3US4EMPPGWYZTL2MTA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:46.292463986 (60h34m17.482762567s) [noise=true , address=104.36.82.8:4972 , piece_id=ER6QEA7NDGF7WKGZQM6KOXDQ72GRJUCOPTB4X4SFEF2VJ74WKTHA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:09:46.292506639 (60h34m17.48280521s) [noise=true , address=104.36.82.10:4078 , piece_id=DWLO4ER732CXF4UNSFBN4SE6BWSHSZBNSGFJ3KY2GP6NPOKAVJWA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:09:46.292559447 (60h34m17.482858018s) [noise=true , address=104.36.82.12:4149 , piece_id=BXUFNBPQ3MP6EUXQ4FJU45EGQ47J7FPK6ALELVQ2ILHOZLLUFL4Q, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:09:46.292748105 (60h34m17.483046684s) [noise=true , address=104.36.82.9:4006 , piece_id=NVAH7SXF2G4FJ6FI7XRCW4RE5ZQEVPV7NCLQYBE6UKWDPZPPO4EA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 139.59.144.22:38176->104.36.82.9:4006: use of closed network connection; write tcp 139.59.144.22:38176->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38176->104.36.82.9:4006: use of closed network connection Jul 27 00:09:46.292781903 (60h34m17.483080488s) [noise=true , address=104.36.82.14:4223 , piece_id=YW7GUOEGIXNZEN33TDQX7IDVJGO4ZGU7GWNKVLIOQ54HBVAYSIDQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:55070->104.36.82.14:4223: use of closed network connection; write tcp 139.59.144.22:55070->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55070->104.36.82.14:4223: use of closed network connection Jul 27 00:09:46.292950155 (60h34m17.483248744s) [seg_pos={0 0}, upload=58061]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 27 00:09:46.399647939 (60h34m17.589946519s) [upload=58061]: Done uploading segments... Jul 27 00:09:46.399697285 (60h34m17.589995882s) [upload=58061]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 3.754950456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 39 out of 60 is failed (threshold=3) [XXXXXXXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X.X] duration: 30.004643067s FAIL (2025-07-27 00:16:52 UTC) orbiter warning: uplink: context deadline exceeded Duration: 30.004696636s core uplink us1 vivint
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [X........XX.X....X......X..X............X.X..X.X............] duration: 28.968658659s FAIL (2025-07-27 00:16:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 28.968966307s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............................X.......................X.X.....] duration: 847.09491ms FAIL (2025-07-27 00:16:33 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 847.127165ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [XXX..X.....XX.X.X......X..X...X.X.X..X..XXX....XX......X..X.] duration: 47.420790834s FAIL (2025-07-27 00:16:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:14:42.538877658 (60h39m13.729176236s) [upload=58146]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:14:42.739636704 (60h39m13.929935299s) [seg_pos={0 0}, upload=58146]: Begin upload segment... Jul 27 00:14:42.739701279 (60h39m13.929999858s) [seg_pos={0 0}, upload=58146]: Done begin upload segment. Jul 27 00:14:42.739741629 (60h39m13.930040209s) [noise=true , address=77.90.31.10:4528 , piece_id=JVJHEJDNGP6GZL2XOW22JJ5NRWY3GMAU4UJ6O6HPFYLTVOBNIYUQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.739750833 (60h39m13.930049415s) [noise=true , address=104.36.82.41:4772 , piece_id=JYQ6P4EYCLWJJV7HZUU6DAFYMBQJPOEJ5NEFAIJVWER3CLWWYKEA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740135209 (60h39m13.930433808s) [noise=true , address=104.36.82.5:4854 , piece_id=6OJS4SYNSIH2VHL7TFQRLTXVGRBFVZA3OIPEA7Y2UQDI7IIUW7QA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740188883 (60h39m13.930487468s) [noise=true , address=79.127.201.215:4356 , piece_id=MWRVNKONLFS574ATTF4ZM2MJGXVPVVJ4UIS6ETYDEQLT22CLIAFA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740449586 (60h39m13.930748172s) [noise=true , address=104.36.82.56:4241 , piece_id=C2TWLPSOM2OWCLJVCE2BIR2RAFFHUX2C3BYDHH4CCSGSXEP7IHDA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740524021 (60h39m13.930822603s) [noise=true , address=77.90.31.13:4644 , piece_id=D3UGBM7OM5OHTDBCFVNVIJGKCUWEM2GRAOL3XKYPGNTJNYO4L23A, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740884918 (60h39m13.931183507s) [noise=true , address=79.127.219.48:4074 , piece_id=2CL3CL3TEXWGBNFJO5X3X32HDDOVIFCBKDDU3APHYTDPE27BRNAQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741260115 (60h39m13.931558709s) [noise=true , address=64.49.13.140:4508 , piece_id=J2KV33QQHCUAVLBSABR4XVY3VXU6ZDS52XCTAEBRCVH2LY5EBCYA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741615447 (60h39m13.931914273s) [noise=true , address=79.127.205.246:4284 , piece_id=VLZRH5XZFEWW4W6LWCZ6ZG3JAQYNBW6CRI6RQT4J376L44KOAFHQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741990455 (60h39m13.932289044s) [noise=true , address=104.36.82.46:4489 , piece_id=T3UH4QBINTILCG4CGXUFS6LNPALQ5IJWLFBFKTDTBEKV7QBNF53Q, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742045465 (60h39m13.932344055s) [noise=true , address=104.36.82.54:4169 , piece_id=S452MUJKZV7RFCYWYVHCV6B5KMVCGUYZ66DIOYHTKNA6VLE7BNSA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742403647 (60h39m13.932702234s) [noise=true , address=77.90.31.11:4579 , piece_id=I5M4EFRTPP6YCGM3RDHCUYYARAR5U2AJNHJQTOFQFP3I5UL3FT3Q, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742852820 (60h39m13.933151405s) [noise=true , address=104.36.82.8:4999 , piece_id=SKWZANHJTKKQPI4DXRAX7Q4W3J5LUCXSXFZWBLPCCHB7NJZ2DNTA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743182186 (60h39m13.933480771s) [noise=true , address=79.127.205.245:4247 , piece_id=QWEM6REHHNRH6ZRKH42ZFNCNSWOTGGPZT5U4O24ANT6FQXSG4MQA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743537911 (60h39m13.933836494s) [noise=true , address=79.127.219.38:4017 , piece_id=AEMM6SRH2QHLOE4FH2EYCSG3PG7KFX7KQYGXJ6XAMIVQPYGT3DMQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743894189 (60h39m13.93419278s) [noise=true , address=104.36.82.47:4547 , piece_id=2K2PFVND2GJY6JQ4WPRXLHY6ATAFLUGZUQD5TXGEFWHZPTHSGZCA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.744216490 (60h39m13.934515072s) [noise=true , address=77.90.31.21:4909 , piece_id=GYN722GGZX7SFTJFNE7JFFQCF5JYDAJMNSUUDZSYT4JDUBCY6ETQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.744506947 (60h39m13.934805518s) [noise=true , address=77.90.31.16:4736 , piece_id=VFM2T2ZVOMXQ5WWHGTACKXZONVP4IKIZA2RCKVZVMQYAZNSUFFJQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:43.313073027 (60h39m14.503371612s) [upload=58146]: Next returned nil segment Jul 27 00:14:43.313095000 (60h39m14.503393588s) [upload=58146]: Waiting for error group managing segments... Jul 27 00:14:43.495135149 (60h39m14.68543372s) [noise=true , address=77.90.31.16:4736 , piece_id=VFM2T2ZVOMXQ5WWHGTACKXZONVP4IKIZA2RCKVZVMQYAZNSUFFJQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.518601307 (60h39m14.708899891s) [noise=true , address=77.90.31.11:4579 , piece_id=I5M4EFRTPP6YCGM3RDHCUYYARAR5U2AJNHJQTOFQFP3I5UL3FT3Q, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.533843430 (60h39m14.724142005s) [noise=true , address=79.127.219.38:4017 , piece_id=AEMM6SRH2QHLOE4FH2EYCSG3PG7KFX7KQYGXJ6XAMIVQPYGT3DMQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.544355195 (60h39m14.734653762s) [noise=true , address=79.127.219.48:4074 , piece_id=2CL3CL3TEXWGBNFJO5X3X32HDDOVIFCBKDDU3APHYTDPE27BRNAQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.987358494 (60h39m15.177657073s) [noise=true , address=79.127.205.246:4284 , piece_id=VLZRH5XZFEWW4W6LWCZ6ZG3JAQYNBW6CRI6RQT4J376L44KOAFHQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:44.088812791 (60h39m15.27911136s) [noise=true , address=79.127.205.245:4247 , piece_id=QWEM6REHHNRH6ZRKH42ZFNCNSWOTGGPZT5U4O24ANT6FQXSG4MQA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:46.408185626 (60h39m17.598484211s) [noise=true , address=77.90.31.13:4644 , piece_id=D3UGBM7OM5OHTDBCFVNVIJGKCUWEM2GRAOL3XKYPGNTJNYO4L23A, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:50.957512682 (60h39m22.147811262s) [noise=true , address=64.49.13.140:4508 , piece_id=J2KV33QQHCUAVLBSABR4XVY3VXU6ZDS52XCTAEBRCVH2LY5EBCYA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:00.579829869 (60h39m31.770128446s) [noise=true , address=104.36.82.56:4241 , piece_id=C2TWLPSOM2OWCLJVCE2BIR2RAFFHUX2C3BYDHH4CCSGSXEP7IHDA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:06.746898478 (60h39m37.937197061s) [noise=true , address=104.36.82.54:4169 , piece_id=S452MUJKZV7RFCYWYVHCV6B5KMVCGUYZ66DIOYHTKNA6VLE7BNSA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:18.131452986 (60h39m49.321751559s) [noise=true , address=79.127.201.215:4356 , piece_id=MWRVNKONLFS574ATTF4ZM2MJGXVPVVJ4UIS6ETYDEQLT22CLIAFA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:25.447346116 (60h39m56.637644705s) [noise=true , address=104.36.82.47:4547 , piece_id=2K2PFVND2GJY6JQ4WPRXLHY6ATAFLUGZUQD5TXGEFWHZPTHSGZCA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:42.261804835 (60h40m13.452103415s) [noise=true , address=104.36.82.5:4854 , piece_id=6OJS4SYNSIH2VHL7TFQRLTXVGRBFVZA3OIPEA7Y2UQDI7IIUW7QA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy, address:104.36.82.5:4854): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.261997725 (60h40m13.452296305s) [noise=true , address=104.36.82.41:4772 , piece_id=JYQ6P4EYCLWJJV7HZUU6DAFYMBQJPOEJ5NEFAIJVWER3CLWWYKEA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:1zKCizjMAKNzq6CvZaVBJmjox6abJzywmtbjCUS1dWcDV2qSip, address:104.36.82.41:4772): protocol: expected piece hash; write tcp 139.59.144.22:60272->104.36.82.41:4772: use of closed network connection; write tcp 139.59.144.22:60272->104.36.82.41:4772: use of closed network connection; EOF Jul 27 00:15:42.262040696 (60h40m13.452339275s) [noise=true , address=77.90.31.10:4528 , piece_id=JVJHEJDNGP6GZL2XOW22JJ5NRWY3GMAU4UJ6O6HPFYLTVOBNIYUQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:15fkvAWei1E5Tn2BYkCavyZ8se6XwPJcmpr7aEsv8sbMtk1n3Q, address:77.90.31.10:4528): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.262053263 (60h40m13.452351835s) [noise=true , address=104.36.82.8:4999 , piece_id=SKWZANHJTKKQPI4DXRAX7Q4W3J5LUCXSXFZWBLPCCHB7NJZ2DNTA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:12Y993kGnB4JusXCcZ4KGyGpRBwoHksVBPzK77ePMw9G7djQEyY, address:104.36.82.8:4999): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.262101355 (60h40m13.452399935s) [noise=true , address=104.36.82.46:4489 , piece_id=T3UH4QBINTILCG4CGXUFS6LNPALQ5IJWLFBFKTDTBEKV7QBNF53Q, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:12YbmXQeh182PYH1AHxvGcw3zW3uiuDsbYPomH3HAPNtjbJSpPw, address:104.36.82.46:4489): protocol: write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection; write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection Jul 27 00:15:42.262242633 (60h40m13.452541213s) [noise=true , address=77.90.31.21:4909 , piece_id=GYN722GGZX7SFTJFNE7JFFQCF5JYDAJMNSUUDZSYT4JDUBCY6ETQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:121RT7tP6czMFAiZ3DjzSSsYgPt3xZXVbXVpkcYxaZYscNiMjL6, address:77.90.31.21:4909): protocol: read tcp 139.59.144.22:48056->77.90.31.21:4909: use of closed network connection; context deadline exceeded Jul 27 00:15:42.262288698 (60h40m13.452587277s) [seg_pos={0 0}, upload=58146]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 27 00:15:42.370689999 (60h40m13.560988579s) [upload=58146]: Done uploading segments... Jul 27 00:15:42.370716421 (60h40m13.561015s) [upload=58146]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 47.420816228s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.X......X..XXX....XXX...X....X.......XX...X....X...XX....X..] duration: 3.09222282s FAIL (2025-07-27 00:16:24 UTC) orbiter warning: context deadline exceeded Duration: 3.092251714s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 37 out of 60 is failed (threshold=3) [XXX.XXXXXXXXXXXXXXX..X.X.X.XX.XXXXX..XX.XX..XXXX.X..........] duration: 28.685068771s FAIL (2025-07-27 00:16:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== d: op Jul 27 00:09:10.565262576 (60h33m41.755561156s) [seg_pos={0 0}, upload=58055]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 27 00:09:10.565365113 (60h33m41.755663693s) [upload=58055]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 27 00:09:10.565370840 (60h33m41.75566942s) [upload=58055]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 27 00:09:10.718568801 (60h33m41.908867373s) [upload=58055]: Done uploading segments... Jul 27 00:09:10.718606181 (60h33m41.90890476s) [upload=58055]: Upload finished. err: <nil> Jul 27 00:09:16.562449298 (60h33m47.752747867s) [upload=58061]: Starting upload Jul 27 00:09:16.562551137 (60h33m47.752849721s) [upload=58061]: Done starting upload Jul 27 00:09:16.562688075 (60h33m47.752986646s) [upload=58061]: Uploading segments... Jul 27 00:09:16.562953051 (60h33m47.753251622s) [upload=58061]: Got next segment. Inline: false Jul 27 00:09:16.563018592 (60h33m47.753317172s) [upload=58061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:09:16.563024426 (60h33m47.753323004s) [upload=58061]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:09:16.674481109 (60h33m47.864779681s) [seg_pos={0 0}, upload=58061]: Begin upload segment... Jul 27 00:09:16.674530332 (60h33m47.864828901s) [seg_pos={0 0}, upload=58061]: Done begin upload segment. Jul 27 00:09:16.674569232 (60h33m47.864867821s) [noise=true , address=104.36.82.10:4078 , piece_id=DWLO4ER732CXF4UNSFBN4SE6BWSHSZBNSGFJ3KY2GP6NPOKAVJWA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.674588419 (60h33m47.864887008s) [noise=true , address=104.36.82.4:4801 , piece_id=CL2IRP22RCTBH3VYQTN7VJ4XPZNBSEJF2O3US4EMPPGWYZTL2MTA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.675107134 (60h33m47.865405733s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMJGODVGCXOJ4TJFPBJKSIEHHQJG2MP7Z7O7B7XCSP7AS4NKAB5A, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.675946131 (60h33m47.866244721s) [noise=true , address=104.36.82.8:4972 , piece_id=ER6QEA7NDGF7WKGZQM6KOXDQ72GRJUCOPTB4X4SFEF2VJ74WKTHA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.676410846 (60h33m47.866709436s) [noise=true , address=45.91.116.11:4118 , piece_id=LKLI6NIPKUYOYQ24N7S7IQ3A43SOM4D5C2AT7LSG4QD463UT3LRA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.677078109 (60h33m47.867376698s) [noise=true , address=77.90.31.17:4771 , piece_id=FLPVNCITXTBBOQRGCKO3B2VQSYAOCPER3CN7DM6DQHM3OW3BXV2Q, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.677277156 (60h33m47.867575737s) [noise=true , address=45.91.116.12:4227 , piece_id=FQOPTSR7UI65EF3UXDTULVD7ZVFWRDRWUC6OSUXSI6VIKEZNXDDQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.677819926 (60h33m47.868118506s) [noise=true , address=104.36.82.12:4149 , piece_id=BXUFNBPQ3MP6EUXQ4FJU45EGQ47J7FPK6ALELVQ2ILHOZLLUFL4Q, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.678383329 (60h33m47.868681921s) [noise=true , address=45.91.116.6:4975 , piece_id=OVDDE74WATWIUHTVB3UTX4P7LD3JO3R2KRDWZW7YU4LCLUKUZQPQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.679073127 (60h33m47.869371716s) [noise=true , address=45.91.116.7:4011 , piece_id=QXNMHUAXWDEBPGP2JHQA6UKWAYN5PKTVRXOLMZM3PBEARYT4VLDA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.679542711 (60h33m47.869841303s) [noise=true , address=45.91.116.10:4082 , piece_id=VW45M6QMGAPSJ7WOTPL5JUDNWGWPNX2TYVLCJAWOTLO3J3DXYLAQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.680001782 (60h33m47.870300373s) [noise=true , address=77.90.31.13:4626 , piece_id=E6D4YLFW2OPSSH3B6GLQIFZPB7RFS2DKHEBHDTMKJOBINVISUMCQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.680556884 (60h33m47.870855472s) [noise=true , address=45.91.116.13:4154 , piece_id=IYW5WRAIUVWDSY6HDN2ZHJM5NRXNKGOVVIH7SCUKIXPOYVMS56BQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.681059039 (60h33m47.871357622s) [noise=true , address=104.36.82.9:4006 , piece_id=NVAH7SXF2G4FJ6FI7XRCW4RE5ZQEVPV7NCLQYBE6UKWDPZPPO4EA, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.681557482 (60h33m47.871856071s) [noise=true , address=104.36.82.14:4223 , piece_id=YW7GUOEGIXNZEN33TDQX7IDVJGO4ZGU7GWNKVLIOQ54HBVAYSIDQ, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:16.682063049 (60h33m47.872361638s) [noise=true , address=45.91.116.14:4190 , piece_id=X3SDGPNCQPWJMJV7KBRYQLC5LEGX6CDAHWLWHRRGOVG5WVTW3C3A, seg_pos={0 0}, upload=58061]: Uploading piece... Jul 27 00:09:17.316057568 (60h33m48.506356156s) [upload=58061]: Next returned nil segment Jul 27 00:09:17.316080757 (60h33m48.506379348s) [upload=58061]: Waiting for error group managing segments... Jul 27 00:09:17.422557859 (60h33m48.61285643s) [noise=true , address=45.91.116.4:4939 , piece_id=ZMJGODVGCXOJ4TJFPBJKSIEHHQJG2MP7Z7O7B7XCSP7AS4NKAB5A, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.424121032 (60h33m48.614419601s) [noise=true , address=45.91.116.6:4975 , piece_id=OVDDE74WATWIUHTVB3UTX4P7LD3JO3R2KRDWZW7YU4LCLUKUZQPQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.445556516 (60h33m48.635855187s) [noise=true , address=45.91.116.7:4011 , piece_id=QXNMHUAXWDEBPGP2JHQA6UKWAYN5PKTVRXOLMZM3PBEARYT4VLDA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.447312205 (60h33m48.637610778s) [noise=true , address=45.91.116.13:4154 , piece_id=IYW5WRAIUVWDSY6HDN2ZHJM5NRXNKGOVVIH7SCUKIXPOYVMS56BQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.455506767 (60h33m48.645805346s) [noise=true , address=45.91.116.10:4082 , piece_id=VW45M6QMGAPSJ7WOTPL5JUDNWGWPNX2TYVLCJAWOTLO3J3DXYLAQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.460800503 (60h33m48.651099087s) [noise=true , address=45.91.116.11:4118 , piece_id=LKLI6NIPKUYOYQ24N7S7IQ3A43SOM4D5C2AT7LSG4QD463UT3LRA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.464762217 (60h33m48.6550608s) [noise=true , address=77.90.31.17:4771 , piece_id=FLPVNCITXTBBOQRGCKO3B2VQSYAOCPER3CN7DM6DQHM3OW3BXV2Q, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.523143661 (60h33m48.713442242s) [noise=true , address=45.91.116.12:4227 , piece_id=FQOPTSR7UI65EF3UXDTULVD7ZVFWRDRWUC6OSUXSI6VIKEZNXDDQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:17.581308187 (60h33m48.771606753s) [noise=true , address=45.91.116.14:4190 , piece_id=X3SDGPNCQPWJMJV7KBRYQLC5LEGX6CDAHWLWHRRGOVG5WVTW3C3A, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:18.942817938 (60h33m50.133116526s) [noise=true , address=77.90.31.13:4626 , piece_id=E6D4YLFW2OPSSH3B6GLQIFZPB7RFS2DKHEBHDTMKJOBINVISUMCQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:26.999729278 (60h33m58.190027866s) [noise=true , address=104.36.82.4:4801 , piece_id=CL2IRP22RCTBH3VYQTN7VJ4XPZNBSEJF2O3US4EMPPGWYZTL2MTA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: <nil> Jul 27 00:09:46.292463986 (60h34m17.482762567s) [noise=true , address=104.36.82.8:4972 , piece_id=ER6QEA7NDGF7WKGZQM6KOXDQ72GRJUCOPTB4X4SFEF2VJ74WKTHA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:09:46.292506639 (60h34m17.48280521s) [noise=true , address=104.36.82.10:4078 , piece_id=DWLO4ER732CXF4UNSFBN4SE6BWSHSZBNSGFJ3KY2GP6NPOKAVJWA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:09:46.292559447 (60h34m17.482858018s) [noise=true , address=104.36.82.12:4149 , piece_id=BXUFNBPQ3MP6EUXQ4FJU45EGQ47J7FPK6ALELVQ2ILHOZLLUFL4Q, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:104.36.82.12:4149): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:09:46.292748105 (60h34m17.483046684s) [noise=true , address=104.36.82.9:4006 , piece_id=NVAH7SXF2G4FJ6FI7XRCW4RE5ZQEVPV7NCLQYBE6UKWDPZPPO4EA, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:104.36.82.9:4006): protocol: write tcp 139.59.144.22:38176->104.36.82.9:4006: use of closed network connection; write tcp 139.59.144.22:38176->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38176->104.36.82.9:4006: use of closed network connection Jul 27 00:09:46.292781903 (60h34m17.483080488s) [noise=true , address=104.36.82.14:4223 , piece_id=YW7GUOEGIXNZEN33TDQX7IDVJGO4ZGU7GWNKVLIOQ54HBVAYSIDQ, seg_pos={0 0}, upload=58061]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 139.59.144.22:55070->104.36.82.14:4223: use of closed network connection; write tcp 139.59.144.22:55070->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:55070->104.36.82.14:4223: use of closed network connection Jul 27 00:09:46.292950155 (60h34m17.483248744s) [seg_pos={0 0}, upload=58061]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 27 00:09:46.399647939 (60h34m17.589946519s) [upload=58061]: Done uploading segments... Jul 27 00:09:46.399697285 (60h34m17.589995882s) [upload=58061]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 28.685104009s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [.X........XX.X....X......X..X............X.X..X.X...........] duration: 48.982346557s FAIL (2025-07-27 00:16:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 48.982369273s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.......................X.X....] duration: 937.074756ms FAIL (2025-07-27 00:15:59 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 937.119436ms core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 39 out of 60 is failed (threshold=3) [XXXXXXXXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X.] duration: 2.395400017s FAIL (2025-07-27 00:15:48 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 2.395434002s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [XXXX..X.....XX.X.X......X..X...X.X.X..X..XXX....XX......X..X] duration: 1m0.110630387s FAIL (2025-07-27 00:15:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== batch item: pb.BatchRequestItem_ObjectBegin Jul 27 00:14:42.538877658 (60h39m13.729176236s) [upload=58146]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 27 00:14:42.739636704 (60h39m13.929935299s) [seg_pos={0 0}, upload=58146]: Begin upload segment... Jul 27 00:14:42.739701279 (60h39m13.929999858s) [seg_pos={0 0}, upload=58146]: Done begin upload segment. Jul 27 00:14:42.739741629 (60h39m13.930040209s) [noise=true , address=77.90.31.10:4528 , piece_id=JVJHEJDNGP6GZL2XOW22JJ5NRWY3GMAU4UJ6O6HPFYLTVOBNIYUQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.739750833 (60h39m13.930049415s) [noise=true , address=104.36.82.41:4772 , piece_id=JYQ6P4EYCLWJJV7HZUU6DAFYMBQJPOEJ5NEFAIJVWER3CLWWYKEA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740135209 (60h39m13.930433808s) [noise=true , address=104.36.82.5:4854 , piece_id=6OJS4SYNSIH2VHL7TFQRLTXVGRBFVZA3OIPEA7Y2UQDI7IIUW7QA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740188883 (60h39m13.930487468s) [noise=true , address=79.127.201.215:4356 , piece_id=MWRVNKONLFS574ATTF4ZM2MJGXVPVVJ4UIS6ETYDEQLT22CLIAFA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740449586 (60h39m13.930748172s) [noise=true , address=104.36.82.56:4241 , piece_id=C2TWLPSOM2OWCLJVCE2BIR2RAFFHUX2C3BYDHH4CCSGSXEP7IHDA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740524021 (60h39m13.930822603s) [noise=true , address=77.90.31.13:4644 , piece_id=D3UGBM7OM5OHTDBCFVNVIJGKCUWEM2GRAOL3XKYPGNTJNYO4L23A, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.740884918 (60h39m13.931183507s) [noise=true , address=79.127.219.48:4074 , piece_id=2CL3CL3TEXWGBNFJO5X3X32HDDOVIFCBKDDU3APHYTDPE27BRNAQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741260115 (60h39m13.931558709s) [noise=true , address=64.49.13.140:4508 , piece_id=J2KV33QQHCUAVLBSABR4XVY3VXU6ZDS52XCTAEBRCVH2LY5EBCYA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741615447 (60h39m13.931914273s) [noise=true , address=79.127.205.246:4284 , piece_id=VLZRH5XZFEWW4W6LWCZ6ZG3JAQYNBW6CRI6RQT4J376L44KOAFHQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.741990455 (60h39m13.932289044s) [noise=true , address=104.36.82.46:4489 , piece_id=T3UH4QBINTILCG4CGXUFS6LNPALQ5IJWLFBFKTDTBEKV7QBNF53Q, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742045465 (60h39m13.932344055s) [noise=true , address=104.36.82.54:4169 , piece_id=S452MUJKZV7RFCYWYVHCV6B5KMVCGUYZ66DIOYHTKNA6VLE7BNSA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742403647 (60h39m13.932702234s) [noise=true , address=77.90.31.11:4579 , piece_id=I5M4EFRTPP6YCGM3RDHCUYYARAR5U2AJNHJQTOFQFP3I5UL3FT3Q, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.742852820 (60h39m13.933151405s) [noise=true , address=104.36.82.8:4999 , piece_id=SKWZANHJTKKQPI4DXRAX7Q4W3J5LUCXSXFZWBLPCCHB7NJZ2DNTA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743182186 (60h39m13.933480771s) [noise=true , address=79.127.205.245:4247 , piece_id=QWEM6REHHNRH6ZRKH42ZFNCNSWOTGGPZT5U4O24ANT6FQXSG4MQA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743537911 (60h39m13.933836494s) [noise=true , address=79.127.219.38:4017 , piece_id=AEMM6SRH2QHLOE4FH2EYCSG3PG7KFX7KQYGXJ6XAMIVQPYGT3DMQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.743894189 (60h39m13.93419278s) [noise=true , address=104.36.82.47:4547 , piece_id=2K2PFVND2GJY6JQ4WPRXLHY6ATAFLUGZUQD5TXGEFWHZPTHSGZCA, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.744216490 (60h39m13.934515072s) [noise=true , address=77.90.31.21:4909 , piece_id=GYN722GGZX7SFTJFNE7JFFQCF5JYDAJMNSUUDZSYT4JDUBCY6ETQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:42.744506947 (60h39m13.934805518s) [noise=true , address=77.90.31.16:4736 , piece_id=VFM2T2ZVOMXQ5WWHGTACKXZONVP4IKIZA2RCKVZVMQYAZNSUFFJQ, seg_pos={0 0}, upload=58146]: Uploading piece... Jul 27 00:14:43.313073027 (60h39m14.503371612s) [upload=58146]: Next returned nil segment Jul 27 00:14:43.313095000 (60h39m14.503393588s) [upload=58146]: Waiting for error group managing segments... Jul 27 00:14:43.495135149 (60h39m14.68543372s) [noise=true , address=77.90.31.16:4736 , piece_id=VFM2T2ZVOMXQ5WWHGTACKXZONVP4IKIZA2RCKVZVMQYAZNSUFFJQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.518601307 (60h39m14.708899891s) [noise=true , address=77.90.31.11:4579 , piece_id=I5M4EFRTPP6YCGM3RDHCUYYARAR5U2AJNHJQTOFQFP3I5UL3FT3Q, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.533843430 (60h39m14.724142005s) [noise=true , address=79.127.219.38:4017 , piece_id=AEMM6SRH2QHLOE4FH2EYCSG3PG7KFX7KQYGXJ6XAMIVQPYGT3DMQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.544355195 (60h39m14.734653762s) [noise=true , address=79.127.219.48:4074 , piece_id=2CL3CL3TEXWGBNFJO5X3X32HDDOVIFCBKDDU3APHYTDPE27BRNAQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:43.987358494 (60h39m15.177657073s) [noise=true , address=79.127.205.246:4284 , piece_id=VLZRH5XZFEWW4W6LWCZ6ZG3JAQYNBW6CRI6RQT4J376L44KOAFHQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:44.088812791 (60h39m15.27911136s) [noise=true , address=79.127.205.245:4247 , piece_id=QWEM6REHHNRH6ZRKH42ZFNCNSWOTGGPZT5U4O24ANT6FQXSG4MQA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:46.408185626 (60h39m17.598484211s) [noise=true , address=77.90.31.13:4644 , piece_id=D3UGBM7OM5OHTDBCFVNVIJGKCUWEM2GRAOL3XKYPGNTJNYO4L23A, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:14:50.957512682 (60h39m22.147811262s) [noise=true , address=64.49.13.140:4508 , piece_id=J2KV33QQHCUAVLBSABR4XVY3VXU6ZDS52XCTAEBRCVH2LY5EBCYA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:00.579829869 (60h39m31.770128446s) [noise=true , address=104.36.82.56:4241 , piece_id=C2TWLPSOM2OWCLJVCE2BIR2RAFFHUX2C3BYDHH4CCSGSXEP7IHDA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:06.746898478 (60h39m37.937197061s) [noise=true , address=104.36.82.54:4169 , piece_id=S452MUJKZV7RFCYWYVHCV6B5KMVCGUYZ66DIOYHTKNA6VLE7BNSA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:18.131452986 (60h39m49.321751559s) [noise=true , address=79.127.201.215:4356 , piece_id=MWRVNKONLFS574ATTF4ZM2MJGXVPVVJ4UIS6ETYDEQLT22CLIAFA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:25.447346116 (60h39m56.637644705s) [noise=true , address=104.36.82.47:4547 , piece_id=2K2PFVND2GJY6JQ4WPRXLHY6ATAFLUGZUQD5TXGEFWHZPTHSGZCA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: <nil> Jul 27 00:15:42.261804835 (60h40m13.452103415s) [noise=true , address=104.36.82.5:4854 , piece_id=6OJS4SYNSIH2VHL7TFQRLTXVGRBFVZA3OIPEA7Y2UQDI7IIUW7QA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:19pxQXokdTJ9W6k5aZrzGZSoAn9p1NqpwbTWMSC6U7ncr8ipwy, address:104.36.82.5:4854): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.261997725 (60h40m13.452296305s) [noise=true , address=104.36.82.41:4772 , piece_id=JYQ6P4EYCLWJJV7HZUU6DAFYMBQJPOEJ5NEFAIJVWER3CLWWYKEA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:1zKCizjMAKNzq6CvZaVBJmjox6abJzywmtbjCUS1dWcDV2qSip, address:104.36.82.41:4772): protocol: expected piece hash; write tcp 139.59.144.22:60272->104.36.82.41:4772: use of closed network connection; write tcp 139.59.144.22:60272->104.36.82.41:4772: use of closed network connection; EOF Jul 27 00:15:42.262040696 (60h40m13.452339275s) [noise=true , address=77.90.31.10:4528 , piece_id=JVJHEJDNGP6GZL2XOW22JJ5NRWY3GMAU4UJ6O6HPFYLTVOBNIYUQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:15fkvAWei1E5Tn2BYkCavyZ8se6XwPJcmpr7aEsv8sbMtk1n3Q, address:77.90.31.10:4528): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.262053263 (60h40m13.452351835s) [noise=true , address=104.36.82.8:4999 , piece_id=SKWZANHJTKKQPI4DXRAX7Q4W3J5LUCXSXFZWBLPCCHB7NJZ2DNTA, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:12Y993kGnB4JusXCcZ4KGyGpRBwoHksVBPzK77ePMw9G7djQEyY, address:104.36.82.8:4999): protocol: expected piece hash; context deadline exceeded; EOF Jul 27 00:15:42.262101355 (60h40m13.452399935s) [noise=true , address=104.36.82.46:4489 , piece_id=T3UH4QBINTILCG4CGXUFS6LNPALQ5IJWLFBFKTDTBEKV7QBNF53Q, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:12YbmXQeh182PYH1AHxvGcw3zW3uiuDsbYPomH3HAPNtjbJSpPw, address:104.36.82.46:4489): protocol: write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection; write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:56754->104.36.82.46:4489: use of closed network connection Jul 27 00:15:42.262242633 (60h40m13.452541213s) [noise=true , address=77.90.31.21:4909 , piece_id=GYN722GGZX7SFTJFNE7JFFQCF5JYDAJMNSUUDZSYT4JDUBCY6ETQ, seg_pos={0 0}, upload=58146]: Done uploading piece. err: ecclient: upload failed (node:121RT7tP6czMFAiZ3DjzSSsYgPt3xZXVbXVpkcYxaZYscNiMjL6, address:77.90.31.21:4909): protocol: read tcp 139.59.144.22:48056->77.90.31.21:4909: use of closed network connection; context deadline exceeded Jul 27 00:15:42.262288698 (60h40m13.452587277s) [seg_pos={0 0}, upload=58146]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 27 00:15:42.370689999 (60h40m13.560988579s) [upload=58146]: Done uploading segments... Jul 27 00:15:42.370716421 (60h40m13.561015s) [upload=58146]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.11066334s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 40 out of 60 is failed (threshold=3) [XXXXXXXXXX.XXXXX..XXXXX..XXXXX..XXXXX..XXXX.XX..X....XX....X] duration: 30.111156541s FAIL (2025-07-27 00:15:41 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.111188237s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.......................X.X...] duration: 846.223955ms FAIL (2025-07-27 00:15:26 UTC) orbiter warning: unable to find key in bucket: sj://uplink/orbiter_us_0/1K Duration: 846.25525ms core uplink us1 vivint
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..X......X..XXX....XXX...X....X.......XX...X....X...XX....X.] duration: 3.948605744s FAIL (2025-07-27 00:15:19 UTC) orbiter warning: context deadline exceeded Duration: 3.948686713s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [..X........XX.X....X......X..X............X.X..X.X..........] duration: 55.521855344s FAIL (2025-07-27 00:15:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 55.522181836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [...X......X..XXX....XXX...X....X.......XX...X....X...XX....X] duration: 30.001449047s FAIL (2025-07-27 00:14:52 UTC) orbiter warning: context deadline exceeded Duration: 30.001478s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)