Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.355280833s FAIL (2025-09-01 03:26:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.355315459s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 926.037027ms FAIL (2025-09-01 03:26:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 926.064543ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.48462616s FAIL (2025-09-01 03:26:03 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.484738263s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 874.451853ms FAIL (2025-09-01 03:25:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 874.489998ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.333137337s FAIL (2025-09-01 03:25:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.333175918s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 897.085563ms FAIL (2025-09-01 03:25:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 897.146204ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 2.336321308s FAIL (2025-09-01 03:24:55 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.33634787s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 818.402541ms FAIL (2025-09-01 03:24:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 818.437286ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.30980416s FAIL (2025-09-01 03:24:39 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.309845316s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.41705817s FAIL (2025-09-01 03:24:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.41709542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 825.215288ms FAIL (2025-09-01 03:24:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 825.242209ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 894.090905ms FAIL (2025-09-01 03:24:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 894.12317ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.693895573s FAIL (2025-09-01 03:24:04 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.693935267s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 1.037948685s FAIL (2025-09-01 03:23:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.037986405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.509091825s FAIL (2025-09-01 03:23:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.50922994s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.349032351s FAIL (2025-09-01 03:23:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.349065878s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.244504811s FAIL (2025-09-01 03:23:33 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.244535432s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.038574014s FAIL (2025-09-01 03:23:16 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.03860438s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.305340462s FAIL (2025-09-01 03:23:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.305375708s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.064549285s FAIL (2025-09-01 03:22:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.064576168s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 867.231088ms FAIL (2025-09-01 03:22:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 867.259684ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.694248111s FAIL (2025-09-01 03:22:43 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.694286544s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.260162865s FAIL (2025-09-01 03:22:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.26022771s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.58993097s FAIL (2025-09-01 03:22:09 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.59020848s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 907.253173ms FAIL (2025-09-01 03:21:54 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 907.281717ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.355000703s FAIL (2025-09-01 03:21:51 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.355034961s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 938.408367ms FAIL (2025-09-01 03:21:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 938.504002ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.551528957s FAIL (2025-09-01 03:21:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.55156889s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.466849931s FAIL (2025-09-01 03:21:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.466909992s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.007664282s FAIL (2025-09-01 03:21:11 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.007700396s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.392682093s FAIL (2025-09-01 03:20:51 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.3927193s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 1.015703839s FAIL (2025-09-01 03:20:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.015748022s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.404342124s FAIL (2025-09-01 03:20:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.404378851s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 925.882209ms FAIL (2025-09-01 03:20:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 925.912708ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.319612237s FAIL (2025-09-01 03:20:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.319647973s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.051126222s FAIL (2025-09-01 03:19:45 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.051157037s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 904.964494ms FAIL (2025-09-01 03:19:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 905.018309ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 855.736164ms FAIL (2025-09-01 03:19:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 855.770561ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 2.591432072s FAIL (2025-09-01 03:19:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.591470367s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.11659635s FAIL (2025-09-01 03:19:04 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.116729764s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 898.204382ms FAIL (2025-09-01 03:18:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 898.232389ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.205032011s FAIL (2025-09-01 03:18:48 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.205072032s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 889.607745ms FAIL (2025-09-01 03:18:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 889.705975ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 1.02795954s FAIL (2025-09-01 03:18:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.028007475s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.417631428s FAIL (2025-09-01 03:18:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 1 02:49:41.922873831 (60h42m2.415306264s) [upload=53475]: Starting upload Sep 1 02:49:41.922959553 (60h42m2.415392007s) [upload=53475]: Done starting upload Sep 1 02:49:41.923118791 (60h42m2.415551223s) [upload=53475]: Uploading segments... Sep 1 02:49:41.923469666 (60h42m2.415902101s) [upload=53475]: Got next segment. Inline: false Sep 1 02:49:41.923550426 (60h42m2.415982879s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:49:41.923557168 (60h42m2.415989613s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 1 02:49:46.033668860 (60h42m6.526101301s) [seg_pos={0 0}, upload=53475]: Begin upload segment... Sep 1 02:49:46.033717692 (60h42m6.526150138s) [seg_pos={0 0}, upload=53475]: Done begin upload segment. Sep 1 02:49:46.033761357 (60h42m6.526193806s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.033769151 (60h42m6.526201594s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034109180 (60h42m6.526541625s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034217641 (60h42m6.526650086s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034350002 (60h42m6.526782443s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.034619848 (60h42m6.527052297s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035190268 (60h42m6.527622715s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035529158 (60h42m6.5279616s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035716491 (60h42m6.528148936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.035939211 (60h42m6.528371662s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036158433 (60h42m6.528590876s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036232646 (60h42m6.528665087s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036495641 (60h42m6.528928083s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036592108 (60h42m6.529024553s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.036751026 (60h42m6.529183471s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.037019577 (60h42m6.529452005s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Uploading piece... Sep 1 02:49:46.647010163 (60h42m7.139442607s) [upload=53475]: Next returned nil segment Sep 1 02:49:46.647034728 (60h42m7.13946718s) [upload=53475]: Waiting for error group managing segments... Sep 1 02:49:46.781745026 (60h42m7.274177455s) [noise=true , address=77.90.31.16:4733 , piece_id=ANYBPA3IMN7NYTWLMG3S5YACGNTDSMP77NYQCEPZJAGWHBSQ5CPA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.782520207 (60h42m7.274952635s) [noise=true , address=45.91.116.11:4117 , piece_id=K6QXHLPARXJMYHD55FLWYLZNF3KLKHQ75PWTRDMC6ZKM37H52ZVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.786905500 (60h42m7.279337942s) [noise=true , address=45.91.116.10:4081 , piece_id=3URLMQ5GXQ3OVUZ62T7E5S26NIBY2GJA5EPQJT5VPM5KDVFG7TYQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791039704 (60h42m7.283472145s) [noise=true , address=45.91.116.9:4261 , piece_id=7BUO4E6AVLPCIQG2QROCFJCLWKHAIJ52N4SPFGM775FB2L6TGDQA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.791903079 (60h42m7.284335514s) [noise=true , address=77.90.31.17:4771 , piece_id=OL5PLZLPAS4AREV66OTAWLOENQBWM5COECL5KQSVRS3P3KU2LL7A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.793794358 (60h42m7.286226786s) [noise=true , address=45.91.116.14:4190 , piece_id=YO3UBWCPPC2RB223LX2DNW3AURCBN44NOPYLS2KVMGRJIRZTQBVQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.797383007 (60h42m7.289815444s) [noise=true , address=77.90.31.10:4517 , piece_id=7PMTSQLYOBDFMPGUG2ZGXEBXKL6FFISO43TBV4246D7E6TZGGLHA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.799853542 (60h42m7.29228598s) [noise=true , address=45.91.116.7:4009 , piece_id=CD6FP3M4SKINZOOFKZPY7NGFOGABQ7AQOCF4UE35WEO2ZTHO54AQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.809549708 (60h42m7.301982138s) [noise=true , address=104.36.82.12:4149 , piece_id=BJZC7KANXP6V7PEXT4PQL73D3WRWD75AGYU3XWSY3UEEOY657LLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.811695208 (60h42m7.304127637s) [noise=true , address=104.36.82.7:4911 , piece_id=AS6PCGUBKM3XHHUSP7WYINZX2DI7WWSIWX73XZWA2TSB35TSJWQQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.820513836 (60h42m7.312946265s) [noise=true , address=104.36.82.6:4875 , piece_id=UKIUVMRCQGTPTTIVVALBK63M5GT5GOP4NP4TOQV7QMYKAMINN3FQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.823258025 (60h42m7.315690461s) [noise=true , address=104.36.82.9:4007 , piece_id=EXAT5CJWAWT5QOSFSO44TV3MIKSTILGGAE5TQQU3IJDKZDKZBU6A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826550293 (60h42m7.318982728s) [noise=true , address=77.90.31.21:4877 , piece_id=UTGZRKQJPY44KD4N5QZNDB2QTCXFOV4EDC7PHKUPZ2YID3PHVOAA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: <nil> Sep 1 02:49:46.826575261 (60h42m7.319007701s) [seg_pos={0 0}, upload=53475]: Segment reached optimal threshold of 13 pieces. Sep 1 02:49:46.826811069 (60h42m7.319243515s) [noise=true , address=45.91.116.13:4154 , piece_id=3MPK2JN4O6DM27BUZOEWYIJSNHTE4KXAGS45GMOIIREBWGG2UOMA, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.826870258 (60h42m7.319302717s) [noise=true , address=77.90.31.15:4698 , piece_id=TXOY4IA6VUPYJVF3DPH7UBGN4UGLXI5G6SUZ6I5ULA5CH3WGZ55A, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 1 02:49:46.827033504 (60h42m7.319465936s) [noise=true , address=104.36.82.2:4730 , piece_id=6MXWXVBPDI5W2LTXB6CCVTP7XH5F26CERKENRJ3L27MVQZBL4PLQ, seg_pos={0 0}, upload=53475]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:42016->104.36.82.2:4730: use of closed network connection Sep 1 02:49:46.827056053 (60h42m7.319488487s) [seg_pos={0 0}, upload=53475]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 1 02:49:46.827202220 (60h42m7.319634663s) [upload=53475]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 1 02:49:46.827218430 (60h42m7.319650873s) [upload=53475]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:23.280405643 (60h42m43.772838085s) [upload=53475]: Done uploading segments... Sep 1 02:50:23.280459241 (60h42m43.772891685s) [upload=53475]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.417662406s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 968.220609ms FAIL (2025-09-01 03:18:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 968.267476ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.204366693s FAIL (2025-09-01 03:17:59 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.204400536s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.467873538s FAIL (2025-09-01 03:17:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.467907348s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 959.408449ms FAIL (2025-09-01 03:17:32 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 959.441949ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 903.338786ms FAIL (2025-09-01 03:17:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Sep 1 02:50:24.295933756 (60h42m44.788366194s) [upload=53484]: Starting upload Sep 1 02:50:24.296009505 (60h42m44.78844195s) [upload=53484]: Done starting upload Sep 1 02:50:24.296078356 (60h42m44.78851079s) [upload=53484]: Uploading segments... Sep 1 02:50:24.296122179 (60h42m44.78855463s) [upload=53484]: Got next segment. Inline: true Sep 1 02:50:24.296129588 (60h42m44.788562031s) [upload=53484]: Waiting for error group managing segments... Sep 1 02:50:24.296173102 (60h42m44.788605532s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 1 02:50:24.296178989 (60h42m44.788611421s) [upload=53484]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 1 02:50:24.296182389 (60h42m44.788614821s) [upload=53484]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 1 02:50:24.395489877 (60h42m44.88792232s) [upload=53484]: Done uploading segments... Sep 1 02:50:24.395505209 (60h42m44.887937652s) [upload=53484]: Upload finished. err: metaclient: Too Many Requests Duration:* 903.366915ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)