Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.380781169s FAIL (2025-08-11 16:17:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.380811002s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X...X.....X..........................X.....] duration: 2.511811266s FAIL (2025-08-11 16:17:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.511839023s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 3.38457395s FAIL (2025-08-11 16:17:10 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.384666658s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.010145342s FAIL (2025-08-11 16:17:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.0101763s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X...X.....X..........................X....] duration: 2.783830536s FAIL (2025-08-11 16:16:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.783863643s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 962.024402ms FAIL (2025-08-11 16:16:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 962.147715ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 3.451534997s FAIL (2025-08-11 16:16:06 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.451572396s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................X...X.....X..........................X...] duration: 2.500058275s FAIL (2025-08-11 16:15:51 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.500095374s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................X...X.....X..........................X..] duration: 2.386793196s FAIL (2025-08-11 16:15:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.38682279s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 3.331154982s FAIL (2025-08-11 16:15:22 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.331183986s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 884.029068ms FAIL (2025-08-11 16:15:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 884.059099ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................X...X.....X..........................X.] duration: 2.529989225s FAIL (2025-08-11 16:15:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 2.53001973s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 3.345061538s FAIL (2025-08-11 16:14:58 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.345088939s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................X...X.....X..........................X] duration: 1.926595809s FAIL (2025-08-11 16:14:19 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 11 16:14:17.717299239 (70h48m39.676396491s) [upload=73786]: Starting upload Aug 11 16:14:17.717358673 (70h48m39.676455924s) [upload=73786]: Done starting upload Aug 11 16:14:17.717634034 (70h48m39.676731279s) [upload=73786]: Uploading segments... Aug 11 16:14:17.717692078 (70h48m39.676789335s) [upload=73786]: Got next segment. Inline: false Aug 11 16:14:17.717718291 (70h48m39.676815536s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 16:14:17.717723395 (70h48m39.67682064s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 16:14:17.918843315 (70h48m39.877940571s) [seg_pos={0 0}, upload=73786]: Begin upload segment... Aug 11 16:14:17.918923453 (70h48m39.878020708s) [seg_pos={0 0}, upload=73786]: Done begin upload segment. Aug 11 16:14:17.918945144 (70h48m39.878042418s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.918972413 (70h48m39.87806967s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919505403 (70h48m39.878602659s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919567702 (70h48m39.878664941s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.919897327 (70h48m39.878994584s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920258278 (70h48m39.879355534s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920648606 (70h48m39.879745869s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.920923744 (70h48m39.880020987s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921002997 (70h48m39.880100252s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921334895 (70h48m39.880432151s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921670742 (70h48m39.880767999s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.921805457 (70h48m39.880902711s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922227136 (70h48m39.881324395s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.922662724 (70h48m39.881759978s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923094391 (70h48m39.882191639s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923218511 (70h48m39.882315757s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923603768 (70h48m39.882701039s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:17.923954015 (70h48m39.883051269s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Uploading piece... Aug 11 16:14:18.496907515 (70h48m40.456004747s) [upload=73786]: Next returned nil segment Aug 11 16:14:18.496931140 (70h48m40.456028398s) [upload=73786]: Waiting for error group managing segments... Aug 11 16:14:18.646868251 (70h48m40.605965502s) [noise=true , address=104.36.82.39:4756 , piece_id=NDWIBHRXPP2YF3BEBQIOFMSHEIVQ2UHQ6NWMYYEB3KWBCSLZH2LQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.646953214 (70h48m40.606050454s) [noise=true , address=104.36.82.41:4766 , piece_id=ISR7Q5BHMMMSTKBITS7CYAKK7XVR7ZAWSJLVEVUWWVL3Z7XLYIKQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.652746123 (70h48m40.611843356s) [noise=true , address=104.36.82.18:4382 , piece_id=REGT3DSGNV53W6LCA7W6JCOOUQIQWTFAGJ6JB3GERXIH3WF5WSUQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.659889491 (70h48m40.618986776s) [noise=true , address=104.36.82.7:4927 , piece_id=ILNCGRXCMHKUKJGYODFVYR5JDXBNTIQ4N2BQ4MWIE4AOZJ4RRBMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.666935072 (70h48m40.626032317s) [noise=true , address=104.36.82.38:4721 , piece_id=TCCAFEJFYXEPRVQWMJV25HMN6LD6PK72OJU5IFIGLP3JIWMPYSJA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.670873811 (70h48m40.629971046s) [noise=true , address=104.36.82.37:4683 , piece_id=6B5EXTUK23F6SZWBV532Y6BZISZWZWZDTRBQAUDPU7OFMSHMSWVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.673314900 (70h48m40.632412138s) [noise=true , address=104.36.82.45:4917 , piece_id=FSJ742RJJFWJUPEO3ZLKQO5BFDJNGCNTYOO5P5C7KYSBQMJQ7VOQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.675602488 (70h48m40.634699726s) [noise=true , address=104.36.82.13:4198 , piece_id=QJDDZRDH356CRDKWF23LMPLBXV4PGXZUXWQK4WU7I44DURZUJYVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.676097040 (70h48m40.635194277s) [noise=true , address=104.36.82.51:4067 , piece_id=TWQBGAEDJOASZWOL7SR6LDC3FTQVQJT4YN7RLFAHQLT6EA5ZGM6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.677442342 (70h48m40.636539574s) [noise=true , address=104.36.82.46:4514 , piece_id=H6XW2ARMQL3JXRBL247ZLIBQQ3BM6X5WEZWW7ROMWDVILH5G3KZQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.685093622 (70h48m40.644190857s) [noise=true , address=104.36.82.60:4340 , piece_id=4R7U3C3ENNCGHDWH3ESYJVLCS4FNEO25WTDS3XQPCZ65BO5SBYSA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.687933210 (70h48m40.647030448s) [noise=true , address=104.36.82.36:4626 , piece_id=H3KIGNWKTE5SLXWLDAUIKKWHRYAUXQF6GKP7DJTPTR3PN363QU4Q, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.771830852 (70h48m40.7309281s) [noise=true , address=104.36.82.44:4900 , piece_id=2LSSX33VK33JAALCPUGJC7PJNWZGDJNHEKL4NSK2FVPZEGXBOWMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879841841 (70h48m40.838939075s) [noise=true , address=79.127.220.101:4069 , piece_id=ELZVQUY5KFILKH4HX353GZAM7ZAJ3OHEYWR2UJD6RC6GAO2RYOMQ, seg_pos={0 0}, upload=73786]: Done uploading piece. err: <nil> Aug 11 16:14:18.879898158 (70h48m40.838995405s) [seg_pos={0 0}, upload=73786]: Segment reached optimal threshold of 14 pieces. Aug 11 16:14:18.880343045 (70h48m40.839440301s) [noise=true , address=104.36.82.5:4863 , piece_id=SQVCPT4QLUQO3JT3RHJQ4A3VRAINNQFCWPSPQPQILPXFTLTXQX4A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ApjPajPXNsWuwWvZ778CTxL33xrFi2cebEDBNHhFa52Nm2aqn): protocol: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:38418->104.36.82.5:4863: use of closed network connection Aug 11 16:14:18.880360506 (70h48m40.839457755s) [noise=true , address=79.127.201.215:4325 , piece_id=LJB4VMLQ4NHRE76UNTVL4PTIE2EE34CGMUUGXAKQJDGW3KZPS5MA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124BC3547CnJ34CsBzFawJz9kFot4tZ6MTuRXa7Jed8JMSTe72D): protocol: expected piece hash; context canceled; EOF Aug 11 16:14:18.880469514 (70h48m40.839566765s) [noise=true , address=64.49.13.155:4913 , piece_id=FC5RSKD4W7KHZZ7U5IUIZ2A7CRLF5RZRW72XWWCNUT5RPGU7KF6A, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oyYUnWDyFo1npaKpBxMG5K31oUzJnTpcU1XHFmKux3MCVUPZV): protocol: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41196->64.49.13.155:4913: use of closed network connection Aug 11 16:14:18.880536299 (70h48m40.839633564s) [noise=true , address=64.49.13.26:4954 , piece_id=UDJPHBX3WRERKRZRADADABMJ2QH3FO7XW2EJ3QGLOQOIHKG54PVA, seg_pos={0 0}, upload=73786]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uBxL7Gg8kYxEpqfa8jkyG7CJKtx8zYD5vcWveLB7cRrb2UpAo): protocol: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:49430->64.49.13.26:4954: use of closed network connection Aug 11 16:14:18.880572809 (70h48m40.839670067s) [seg_pos={0 0}, upload=73786]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 11 16:14:18.880671162 (70h48m40.839768417s) [upload=73786]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 11 16:14:18.880677404 (70h48m40.839774673s) [upload=73786]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 16:14:19.049787422 (70h48m41.008884662s) [upload=73786]: Done uploading segments... Aug 11 16:14:19.049809030 (70h48m41.008906264s) [upload=73786]: Upload finished. err: <nil> Duration: 1.926627583s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 950.710544ms FAIL (2025-08-11 16:14:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 950.740615ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 3.148759064s FAIL (2025-08-11 16:13:48 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.148786669s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X...X.....X..........................] duration: 2.359329011s FAIL (2025-08-11 16:13:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.359364193s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 874.00243ms FAIL (2025-08-11 16:13:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 874.032396ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X...X.....X.........................] duration: 2.686691716s FAIL (2025-08-11 16:13:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.686732913s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 905.722607ms FAIL (2025-08-11 16:12:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 905.750576ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 3.13660555s FAIL (2025-08-11 16:12:35 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.136703432s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.011267683s FAIL (2025-08-11 16:12:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.011304294s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X...X.....X........................] duration: 2.346564549s FAIL (2025-08-11 16:12:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.34660103s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 3.157422681s FAIL (2025-08-11 16:11:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.157475471s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X...X.....X.......................] duration: 2.445903066s FAIL (2025-08-11 16:11:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.445932731s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 1.119145536s FAIL (2025-08-11 16:11:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.119190244s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.347601752s FAIL (2025-08-11 16:11:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.347636255s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X...X.....X......................] duration: 2.643654953s FAIL (2025-08-11 16:10:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.643685455s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 3.325954453s FAIL (2025-08-11 16:10:49 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.325986197s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 850.270349ms FAIL (2025-08-11 16:09:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 850.29815ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 3.078846629s FAIL (2025-08-11 16:09:56 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.078884263s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X...X.....X.....................] duration: 2.376337976s FAIL (2025-08-11 16:09:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.376368765s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 3.381553504s FAIL (2025-08-11 16:09:31 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.381585393s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 845.138893ms FAIL (2025-08-11 16:09:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 845.172973ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 3.188657456s FAIL (2025-08-11 16:08:46 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.188748573s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X...X.....X....................] duration: 2.33301495s FAIL (2025-08-11 16:08:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.333050174s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 1.037460255s FAIL (2025-08-11 16:08:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.03748945s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 3.207320531s FAIL (2025-08-11 16:08:28 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.207354551s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X...X.....X...................] duration: 2.538214908s FAIL (2025-08-11 16:08:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.538245226s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 3.602148989s FAIL (2025-08-11 16:08:01 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.602186824s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 869.657873ms FAIL (2025-08-11 16:07:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 869.701167ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 3.488876646s FAIL (2025-08-11 16:07:45 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.488916266s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X...X.....X..................] duration: 2.801667484s FAIL (2025-08-11 16:07:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.801701868s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 3.379389814s FAIL (2025-08-11 16:07:24 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.379432414s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X...X.....X.................] duration: 2.54455845s FAIL (2025-08-11 16:07:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.544658933s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 864.632935ms FAIL (2025-08-11 16:06:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 864.655493ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................X...X.....X................] duration: 2.707114978s FAIL (2025-08-11 16:06:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:55:43.102563599 (70h30m5.061660847s) [upload=73463]: Starting upload Aug 11 15:55:43.102656294 (70h30m5.061753541s) [upload=73463]: Done starting upload Aug 11 15:55:43.102814265 (70h30m5.061911504s) [upload=73463]: Uploading segments... Aug 11 15:55:43.102923111 (70h30m5.06202036s) [upload=73463]: Got next segment. Inline: false Aug 11 15:55:43.102970683 (70h30m5.062067929s) [upload=73463]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:55:43.102978904 (70h30m5.06207615s) [upload=73463]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 11 15:55:43.194308217 (70h30m5.153405474s) [upload=73463]: Next segment err: metaclient: Too Many Requests Aug 11 15:55:43.194338881 (70h30m5.153436138s) [upload=73463]: Done uploading segments... Aug 11 15:55:43.194378381 (70h30m5.153475628s) [upload=73463]: Upload finished. err: metaclient: Too Many Requests Duration: 2.707149725s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 3.291044608s FAIL (2025-08-11 16:06:35 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.291084127s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 3.187678039s FAIL (2025-08-11 16:06:12 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 3.187729645s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 844.165304ms FAIL (2025-08-11 16:05:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 15:37:52.154333504 (70h12m14.113430751s) [upload=73146]: Starting upload Aug 11 15:37:52.154387892 (70h12m14.11348513s) [upload=73146]: Done starting upload Aug 11 15:37:52.154469382 (70h12m14.11356663s) [upload=73146]: Uploading segments... Aug 11 15:37:52.154603604 (70h12m14.113700863s) [upload=73146]: Got next segment. Inline: true Aug 11 15:37:52.154614238 (70h12m14.113711493s) [upload=73146]: Waiting for error group managing segments... Aug 11 15:37:52.154704774 (70h12m14.113802028s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 11 15:37:52.154733324 (70h12m14.113830574s) [upload=73146]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 11 15:37:52.154738270 (70h12m14.113835518s) [upload=73146]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 11 15:37:52.245517230 (70h12m14.204614492s) [upload=73146]: Done uploading segments... Aug 11 15:37:52.245597137 (70h12m14.204694395s) [upload=73146]: Upload finished. err: metaclient: Too Many Requests Duration:* 844.197378ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)