Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
749.548385ms
FAIL (2025-06-24 15:37:15 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 749.580908ms
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
3.197143553s
FAIL (2025-06-24 15:37:05 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.197179719s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............X....................X.......................] duration:
2.631654838s
FAIL (2025-06-24 15:36:53 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.63168997s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
904.534883ms
FAIL (2025-06-24 15:36:53 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 904.577342ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
868.065116ms
FAIL (2025-06-24 15:36:50 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 868.099089ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
3.595976281s
FAIL (2025-06-24 15:36:31 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.596012215s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
849.204312ms
FAIL (2025-06-24 15:36:22 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 849.232398ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
3.073146148s
FAIL (2025-06-24 15:35:56 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.073173651s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................X....................X......................] duration:
2.39828368s
FAIL (2025-06-24 15:35:51 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.398336369s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
888.769741ms
FAIL (2025-06-24 15:35:49 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 888.812849ms
core
uplink
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
878.014993ms
FAIL (2025-06-24 15:35:49 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 878.086947ms
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.285341055s
FAIL (2025-06-24 15:35:04 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.285460851s
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
992.811011ms
FAIL (2025-06-24 15:35:03 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 992.872911ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................X....................X.....................] duration:
2.414255226s
FAIL (2025-06-24 15:34:49 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.414283775s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
3.03007991s
FAIL (2025-06-24 15:34:49 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.030195122s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
3.115565263s
FAIL (2025-06-24 15:34:08 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.115605487s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
668.988001ms
FAIL (2025-06-24 15:33:59 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 669.02033ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................X....................X....................] duration:
2.669023525s
FAIL (2025-06-24 15:33:58 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.669064493s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
875.039943ms
FAIL (2025-06-24 15:33:56 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 875.078494ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.964003428s
FAIL (2025-06-24 15:33:19 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 2.964028443s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...................X....................X...................] duration:
2.390913514s
FAIL (2025-06-24 15:33:14 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.391026759s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
924.049511ms
FAIL (2025-06-24 15:33:14 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 924.099877ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
866.440922ms
FAIL (2025-06-24 15:32:53 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 866.471309ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....................X....................X..................] duration:
2.386238014s
FAIL (2025-06-24 15:32:42 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.386278345s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
3.325422661s
FAIL (2025-06-24 15:32:16 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.3257246s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
653.796543ms
FAIL (2025-06-24 15:32:16 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 653.823604ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....................X....................X.................] duration:
2.661851923s
FAIL (2025-06-24 15:32:02 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.661949176s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
651.898022ms
FAIL (2025-06-24 15:31:58 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 651.92825ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
859.484779ms
FAIL (2025-06-24 15:31:57 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 859.518108ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
3.623560373s
FAIL (2025-06-24 15:31:09 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.623588133s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......................X....................X................] duration:
2.519387715s
FAIL (2025-06-24 15:31:01 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.519422511s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
827.64434ms
FAIL (2025-06-24 15:30:59 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 827.679147ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
929.421496ms
FAIL (2025-06-24 15:30:53 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 929.44756ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
940.740079ms
FAIL (2025-06-24 15:30:37 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 940.770642ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......................X....................X...............] duration:
2.462824008s
FAIL (2025-06-24 15:30:21 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.462852472s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
645.837038ms
FAIL (2025-06-24 15:30:10 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 645.874294ms
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
3.926310916s
FAIL (2025-06-24 15:30:03 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.926391882s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
856.440281ms
FAIL (2025-06-24 15:29:33 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 856.466663ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
850.051763ms
FAIL (2025-06-24 15:29:30 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 850.085148ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........................X....................X..............] duration:
2.544027012s
FAIL (2025-06-24 15:29:19 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.544067579s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.262928053s
FAIL (2025-06-24 15:29:09 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.26295717s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
3.654473564s
FAIL (2025-06-24 15:29:04 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.654509849s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
1.007569275s
FAIL (2025-06-24 15:29:00 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 1.007879441s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.........................X....................X.............] duration:
2.425772688s
FAIL (2025-06-24 15:28:40 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.425831732s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.321707105s
FAIL (2025-06-24 15:28:35 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.321736461s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
854.833444ms
FAIL (2025-06-24 15:28:31 UTC)
orbiter warning: ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== load=23785]: Done uploading segments... Jun 24 14:46:51.013079804 (29h35m50.896493421s) [upload=23785]: Upload finished. err: <nil> Jun 24 14:48:04.670200767 (29h37m4.55361441s) [upload=23802]: Starting upload Jun 24 14:48:04.670299281 (29h37m4.55371292s) [upload=23802]: Done starting upload Jun 24 14:48:04.670387454 (29h37m4.553801073s) [upload=23802]: Uploading segments... Jun 24 14:48:04.670444140 (29h37m4.553857782s) [upload=23802]: Got next segment. Inline: true Jun 24 14:48:04.670450277 (29h37m4.553863916s) [upload=23802]: Waiting for error group managing segments... Jun 24 14:48:04.670510501 (29h37m4.55392414s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:04.670517634 (29h37m4.55393128s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:04.670521127 (29h37m4.553934768s) [upload=23802]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:04.826718303 (29h37m4.710131939s) [upload=23802]: Done uploading segments... Jun 24 14:48:04.826743702 (29h37m4.710157334s) [upload=23802]: Upload finished. err: <nil> Jun 24 14:48:54.492571378 (29h37m54.375984994s) [upload=23812]: Starting upload Jun 24 14:48:54.492641407 (29h37m54.376055033s) [upload=23812]: Done starting upload Jun 24 14:48:54.492706738 (29h37m54.376120356s) [upload=23812]: Uploading segments... Jun 24 14:48:54.492750211 (29h37m54.376163846s) [upload=23812]: Got next segment. Inline: true Jun 24 14:48:54.492756143 (29h37m54.376169768s) [upload=23812]: Waiting for error group managing segments... Jun 24 14:48:54.492807790 (29h37m54.376221417s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:48:54.492813602 (29h37m54.376227228s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:48:54.492817214 (29h37m54.376230834s) [upload=23812]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:48:54.669542420 (29h37m54.552956052s) [upload=23812]: Done uploading segments... Jun 24 14:48:54.669577365 (29h37m54.552990991s) [upload=23812]: Upload finished. err: <nil> Jun 24 14:49:29.968257553 (29h38m29.85167119s) [upload=23819]: Starting upload Jun 24 14:49:29.968402712 (29h38m29.851816349s) [upload=23819]: Done starting upload Jun 24 14:49:29.968532806 (29h38m29.851946425s) [upload=23819]: Uploading segments... Jun 24 14:49:29.968608429 (29h38m29.852022063s) [upload=23819]: Got next segment. Inline: true Jun 24 14:49:29.968615133 (29h38m29.85202877s) [upload=23819]: Waiting for error group managing segments... Jun 24 14:49:29.968669064 (29h38m29.852082702s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:49:29.968691751 (29h38m29.852105393s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:49:29.968695328 (29h38m29.852108958s) [upload=23819]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:49:30.118887640 (29h38m30.002301306s) [upload=23819]: Done uploading segments... Jun 24 14:49:30.118914877 (29h38m30.002328496s) [upload=23819]: Upload finished. err: <nil> Jun 24 14:50:34.548035510 (29h39m34.431449142s) [upload=23836]: Starting upload Jun 24 14:50:34.548225118 (29h39m34.43163875s) [upload=23836]: Done starting upload Jun 24 14:50:34.548390577 (29h39m34.43180421s) [upload=23836]: Uploading segments... Jun 24 14:50:34.548485672 (29h39m34.431899315s) [upload=23836]: Got next segment. Inline: true Jun 24 14:50:34.548494122 (29h39m34.431907756s) [upload=23836]: Waiting for error group managing segments... Jun 24 14:50:34.548551324 (29h39m34.431965014s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:34.548559214 (29h39m34.431972842s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:34.548563721 (29h39m34.43197735s) [upload=23836]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:34.775268630 (29h39m34.658682265s) [upload=23836]: Done uploading segments... Jun 24 14:50:34.775294922 (29h39m34.658708557s) [upload=23836]: Upload finished. err: <nil> Jun 24 14:50:54.672429460 (29h39m54.555843093s) [upload=23841]: Starting upload Jun 24 14:50:54.672553019 (29h39m54.555966645s) [upload=23841]: Done starting upload Jun 24 14:50:54.672649135 (29h39m54.556062764s) [upload=23841]: Uploading segments... Jun 24 14:50:54.672735125 (29h39m54.556148762s) [upload=23841]: Got next segment. Inline: true Jun 24 14:50:54.672742618 (29h39m54.55615623s) [upload=23841]: Waiting for error group managing segments... Jun 24 14:50:54.672813780 (29h39m54.556227414s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:50:54.672823806 (29h39m54.556237442s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:50:54.672827674 (29h39m54.556241309s) [upload=23841]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:50:54.820997630 (29h39m54.704411266s) [upload=23841]: Done uploading segments... Jun 24 14:50:54.821037983 (29h39m54.704451615s) [upload=23841]: Upload finished. err: <nil> Jun 24 14:51:37.730047463 (29h40m37.613461094s) [upload=23851]: Starting upload Jun 24 14:51:37.730155581 (29h40m37.613569208s) [upload=23851]: Done starting upload Jun 24 14:51:37.730270794 (29h40m37.61368442s) [upload=23851]: Uploading segments... Jun 24 14:51:37.730443275 (29h40m37.61385691s) [upload=23851]: Got next segment. Inline: true Jun 24 14:51:37.730454055 (29h40m37.613867702s) [upload=23851]: Waiting for error group managing segments... Jun 24 14:51:37.730541235 (29h40m37.613954866s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:51:37.730565021 (29h40m37.613978655s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:51:37.730572760 (29h40m37.613986394s) [upload=23851]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:51:37.881769478 (29h40m37.765183114s) [upload=23851]: Done uploading segments... Jun 24 14:51:37.881793269 (29h40m37.765206896s) [upload=23851]: Upload finished. err: <nil> Jun 24 14:52:32.960954350 (29h41m32.844367968s) [upload=23860]: Starting upload Jun 24 14:52:32.961065924 (29h41m32.844479542s) [upload=23860]: Done starting upload Jun 24 14:52:32.961156300 (29h41m32.844569917s) [upload=23860]: Uploading segments... Jun 24 14:52:32.961243963 (29h41m32.8446576s) [upload=23860]: Got next segment. Inline: true Jun 24 14:52:32.961251905 (29h41m32.844665536s) [upload=23860]: Waiting for error group managing segments... Jun 24 14:52:32.961320768 (29h41m32.844734394s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:52:32.961326194 (29h41m32.844739819s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:52:32.961328741 (29h41m32.844742368s) [upload=23860]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:52:33.120149955 (29h41m33.003563587s) [upload=23860]: Done uploading segments... Jun 24 14:52:33.120176412 (29h41m33.00359003s) [upload=23860]: Upload finished. err: <nil> Jun 24 14:53:26.501683969 (29h42m26.385097605s) [upload=23870]: Starting upload Jun 24 14:53:26.501788189 (29h42m26.385201825s) [upload=23870]: Done starting upload Jun 24 14:53:26.501960217 (29h42m26.38537385s) [upload=23870]: Uploading segments... Jun 24 14:53:26.502058530 (29h42m26.385472167s) [upload=23870]: Got next segment. Inline: true Jun 24 14:53:26.502065718 (29h42m26.385479354s) [upload=23870]: Waiting for error group managing segments... Jun 24 14:53:26.502130639 (29h42m26.385544277s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:53:26.502136954 (29h42m26.38555059s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:53:26.502140446 (29h42m26.385554083s) [upload=23870]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:53:26.650852170 (29h42m26.534265785s) [upload=23870]: Done uploading segments... Jun 24 14:53:26.650872362 (29h42m26.534285977s) [upload=23870]: Upload finished. err: <nil> Jun 24 14:54:17.347412693 (29h43m17.230826326s) [upload=23880]: Starting upload Jun 24 14:54:17.347484128 (29h43m17.230897743s) [upload=23880]: Done starting upload Jun 24 14:54:17.347573574 (29h43m17.23098719s) [upload=23880]: Uploading segments... Jun 24 14:54:17.347633079 (29h43m17.231046716s) [upload=23880]: Got next segment. Inline: true Jun 24 14:54:17.347640639 (29h43m17.231054257s) [upload=23880]: Waiting for error group managing segments... Jun 24 14:54:17.347691733 (29h43m17.23110535s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:17.347697740 (29h43m17.231111359s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:17.347701364 (29h43m17.231114993s) [upload=23880]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:54:17.510929153 (29h43m17.394342777s) [upload=23880]: Done uploading segments... Jun 24 14:54:17.510982874 (29h43m17.394396512s) [upload=23880]: Upload finished. err: <nil> Jun 24 14:54:59.993425635 (29h43m59.876839261s) [upload=23889]: Starting upload Jun 24 14:54:59.993521148 (29h43m59.876934774s) [upload=23889]: Done starting upload Jun 24 14:54:59.993596246 (29h43m59.877009864s) [upload=23889]: Uploading segments... Jun 24 14:54:59.993660971 (29h43m59.87707463s) [upload=23889]: Got next segment. Inline: true Jun 24 14:54:59.993668009 (29h43m59.877081635s) [upload=23889]: Waiting for error group managing segments... Jun 24 14:54:59.993751260 (29h43m59.877164887s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Jun 24 14:54:59.993757594 (29h43m59.877171225s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Jun 24 14:54:59.993761664 (29h43m59.877175291s) [upload=23889]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Jun 24 14:55:00.142996318 (29h44m0.026409955s) [upload=23889]: Done uploading segments... Jun 24 14:55:00.143021139 (29h44m0.026434767s) [upload=23889]: Upload finished. err: <nil>
Duration: 854.87206ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
3.073294001s
FAIL (2025-06-24 15:28:25 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 3.07333214s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........................X....................X............] duration:
2.487085473s
FAIL (2025-06-24 15:28:17 UTC)
orbiter warning:
======ERROR======
uplink: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
======LOGS======
(30h9m57.569988099s) [upload=24239]: Got next segment. Inline: false
Jun 24 15:20:57.686619305 (30h9m57.570032934s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 15:20:57.686638888 (30h9m57.570052517s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 15:20:57.928224467 (30h9m57.811638102s) [seg_pos={0 0}, upload=24239]: Begin upload segment...
Jun 24 15:20:57.928301679 (30h9m57.811715319s) [seg_pos={0 0}, upload=24239]: Done begin upload segment.
Jun 24 15:20:57.928337777 (30h9m57.811751414s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928406629 (30h9m57.811820265s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.928863827 (30h9m57.812277463s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929025477 (30h9m57.81243913s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929286116 (30h9m57.812699754s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929564621 (30h9m57.812978257s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.929730217 (30h9m57.813143832s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930153445 (30h9m57.813567083s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930242518 (30h9m57.813656153s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930630185 (30h9m57.81404382s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.930638645 (30h9m57.814052284s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931017196 (30h9m57.814430834s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931132509 (30h9m57.814546156s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931366452 (30h9m57.81478014s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931564988 (30h9m57.814978626s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.931785078 (30h9m57.815198715s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932199806 (30h9m57.815613443s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:57.932597121 (30h9m57.816010761s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Uploading piece...
Jun 24 15:20:58.481271254 (30h9m58.364684873s) [upload=24239]: Next returned nil segment
Jun 24 15:20:58.481353797 (30h9m58.364767428s) [upload=24239]: Waiting for error group managing segments...
Jun 24 15:20:58.655262482 (30h9m58.53867612s) [noise=true , address=104.36.82.42:6802 , piece_id=7OSAF24B3TPOSYMJWZWDFUO6UBKNYX3ZPIGUY2SYAEZ4G4G4DYFQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.657655175 (30h9m58.541068803s) [noise=true , address=77.90.31.16:4743 , piece_id=3DSZF64GBM22EGJKQROQGPLGEHKVNW34JSHOXUDF74RQIEJKESIQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.662068056 (30h9m58.545481674s) [noise=true , address=79.127.219.48:4081 , piece_id=VQIGSE2XEF6HBXVW4VQDZLCNTBF5TZGVJZXBUADXBLVWRYOM5WZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.666971693 (30h9m58.550385332s) [noise=true , address=104.36.82.55:4199 , piece_id=D4ZEWYMPJV7AWA5N25XBNSONSV7ZDRBQZ5CY325NI763DYGDYUWA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.673154612 (30h9m58.556568247s) [noise=true , address=104.36.82.3:4776 , piece_id=EQSOKKZXG4LZJ4DVRRZ5YZ6HCZ6SZ55PH47OATBJAWZZK26KRSHQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.677822189 (30h9m58.561235808s) [noise=true , address=77.90.31.21:4912 , piece_id=ACMFKAM6XWOBQ3SQ7KJFLBFW2FK3ZET7EB7UYOP4RX4UYXK3QRLQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.684887890 (30h9m58.568301504s) [noise=true , address=104.36.82.41:4795 , piece_id=75DNQGGA47MHOAS2X4DKCHMU6GB3NLCYK2WB7RZM53S3PRSR35RA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.690528248 (30h9m58.573941867s) [noise=true , address=104.36.82.12:4166 , piece_id=TJ5O4LAAUC25VCCVXPTFJEF2UNRCFMNE5EV4XTC7RMNH3OX3GKJA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.701981027 (30h9m58.58539464s) [noise=true , address=104.36.82.56:4246 , piece_id=75VOW7N366ZHGJIEVBFXA3XBDQ5CRF5BHECIITMBNZ5YF6HX2WVA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.709947868 (30h9m58.593361508s) [noise=true , address=104.36.82.53:4129 , piece_id=A6HRILO4AOFP2DXL2N36SYIMNQI33EUFKV6O4YLVGMT6IAC2WFHA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.713191677 (30h9m58.596605305s) [noise=true , address=104.36.82.34:4574 , piece_id=HWNI73D3I3RRRTOXA6YL7DAXR2NOPEUMMU77JPEWCSRHF7QW4SXQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.729487718 (30h9m58.612901353s) [noise=true , address=104.36.82.10:4096 , piece_id=GOB7J2EY6YDOGXAQ3K2AY6OKWGDOVMZQVKA4ZD4CKTTOHPYPBNNA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.749570660 (30h9m58.632984293s) [noise=true , address=45.91.116.8:4062 , piece_id=G6AM54EJGNC446AJ72XGH2UTF27O5H5WV2MXLHYTGXQQDGK7OMPQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846275930 (30h9m58.729689563s) [noise=true , address=79.127.220.101:6061 , piece_id=F2EHY2WPVGI5AJYTI7RAKL7TIQEAEXNPQ5KYFWEOH4PDTWCBS5FA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: <nil>
Jun 24 15:20:58.846340737 (30h9m58.729754371s) [seg_pos={0 0}, upload=24239]: Segment reached optimal threshold of 14 pieces.
Jun 24 15:20:58.846632560 (30h9m58.730046195s) [noise=true , address=104.36.82.14:4232 , piece_id=Z73NGQAPCHA5E5OLEFDB5765FQOKDZ6554V7FE5QEC4UBEJJPCZQ, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jWewg226E4BrBxqcK38n4ZjHx3toQeNdbc5HeU2EoykztXwyX): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846836814 (30h9m58.730250441s) [noise=true , address=64.49.13.170:4719 , piece_id=QS7G3PAFGIDQL7CYO3YOYCPZVFRHOD57U4NBLJMQBBOHERVTEZXA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126nRFki7uLyG9xwY8t6BeAumn8ME2HwccdC2EAwsiEV1p2juZ9): protocol: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection; piecestore: piecestore close: write tcp 139.59.144.22:41510->64.49.13.170:4719: use of closed network connection
Jun 24 15:20:58.846884087 (30h9m58.730297723s) [noise=true , address=156.146.43.225:4467 , piece_id=3ASI2Q5YZ3OHFU6NCIK4OWMGXNSFVK3LXBG4O252I3FOGMJ7AVCA, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12eTYJjTHAyM7gvmxrtzHMjFxCCQRzs6fgkVX7Rjj1ze9nVXjwr): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846899396 (30h9m58.730313025s) [noise=true , address=79.127.220.97:4370 , piece_id=GQVHIQNND7U4WQYK5CVGD4VFRGB6CAUSKCGE22VAZK3CTNRF6N3Q, seg_pos={0 0}, upload=24239]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H9d6LpjXV9CHf3BenRN8M1uYNMF6ULNFnZzprJZsktsXCdoQb): protocol: expected piece hash; context canceled; EOF
Jun 24 15:20:58.846918006 (30h9m58.730331619s) [seg_pos={0 0}, upload=24239]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jun 24 15:20:58.847033031 (30h9m58.73044668s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 15:20:58.847051227 (30h9m58.730464859s) [upload=24239]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 15:21:38.513456613 (30h10m38.396870249s) [upload=24239]: Done uploading segments...
Jun 24 15:21:38.513493889 (30h10m38.396907528s) [upload=24239]: Upload finished. err: metaclient: manager closed: closed: read tcp 139.59.144.22:34230->34.150.199.48:7777: read: connection reset by peer
Duration: 2.48711636s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
716.817129ms
FAIL (2025-06-24 15:28:11 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 716.842439ms
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_eu_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.361314787s
FAIL (2025-06-24 15:28:10 UTC)
orbiter warning:
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
e15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585387802 (29h33m30.468801424s) [noise=true , address=45.91.116.4:4937 , piece_id=E3X6JZF54EUUPOZZUOTHJDOWINXUKD55QV2Y2XNIT2UAEERDMTFA, seg_pos={0 0}, upload=23751]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: expected piece hash; context canceled; EOF
Jun 24 14:44:30.585411686 (29h33m30.468825314s) [seg_pos={0 0}, upload=23751]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:44:30.585539140 (29h33m30.468952759s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:44:30.585546106 (29h33m30.468959722s) [upload=23751]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:44:30.740865530 (29h33m30.624279165s) [upload=23751]: Done uploading segments...
Jun 24 14:44:30.740886576 (29h33m30.624300193s) [upload=23751]: Upload finished. err: <nil>
Jun 24 14:45:03.860426310 (29h34m3.743839928s) [upload=23763]: Starting upload
Jun 24 14:45:03.860507701 (29h34m3.74392133s) [upload=23763]: Done starting upload
Jun 24 14:45:03.860622755 (29h34m3.744036374s) [upload=23763]: Uploading segments...
Jun 24 14:45:03.861015793 (29h34m3.744429427s) [upload=23763]: Got next segment. Inline: false
Jun 24 14:45:03.861094807 (29h34m3.744508425s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 24 14:45:03.861101687 (29h34m3.744515317s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 24 14:45:03.969869040 (29h34m3.85328269s) [seg_pos={0 0}, upload=23763]: Begin upload segment...
Jun 24 14:45:03.969975171 (29h34m3.853388803s) [seg_pos={0 0}, upload=23763]: Done begin upload segment.
Jun 24 14:45:03.970015949 (29h34m3.853429588s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970051882 (29h34m3.853465514s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970657032 (29h34m3.854070671s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.970931353 (29h34m3.854344971s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971181589 (29h34m3.854595226s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.971562644 (29h34m3.854976283s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972084815 (29h34m3.855498456s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972110205 (29h34m3.855523835s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972460478 (29h34m3.855874114s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972625384 (29h34m3.856039008s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.972812157 (29h34m3.856225794s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973135313 (29h34m3.856548946s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973199500 (29h34m3.856613138s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.973544617 (29h34m3.856958249s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.978681546 (29h34m3.862095186s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:03.979259735 (29h34m3.86267338s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Uploading piece...
Jun 24 14:45:04.611228750 (29h34m4.49464237s) [upload=23763]: Next returned nil segment
Jun 24 14:45:04.611339814 (29h34m4.494753517s) [upload=23763]: Waiting for error group managing segments...
Jun 24 14:45:04.730086577 (29h34m4.613500192s) [noise=true , address=104.36.82.15:4259 , piece_id=GOE3R3YONCILBOOYOZJ7UI75N34WIGOKNOGLI7H2Z3T7LTBODSZQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.730778103 (29h34m4.614191733s) [noise=true , address=45.91.116.10:4083 , piece_id=XVVBBGJJ2S23AV44DJPTXQLKRTXDTRDJOBPMGBTFSX5COD3DRSPA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.733275401 (29h34m4.616689015s) [noise=true , address=45.91.116.5:5297 , piece_id=EQQH2NFE5QU2GBGICVAVADWKV7IEZ7VLDZ3AZSJFEWWALWO3IU2Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.740658584 (29h34m4.624072263s) [noise=true , address=77.90.31.10:4518 , piece_id=ZUWO7JW3PUZCV3EM5A6DNSKCEOUUELTU3CORHW5GXTT6TV6QAI5A, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741248300 (29h34m4.624661929s) [noise=true , address=45.91.116.12:5225 , piece_id=PQI2SNNDF2W7CM7GXIPFRVAFAXT7DYP5S5I572HSZR47JS3XKB5Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741292515 (29h34m4.624706144s) [noise=true , address=45.91.116.13:4155 , piece_id=BRLGLADB7AQVQYC7HEHITXGXFLDODFMNZ4BPNCP5B2U2NKIUCP4Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.741822088 (29h34m4.625235722s) [noise=true , address=45.91.116.7:4010 , piece_id=3JRLL56HTHXGQRQ72WVHVW4T2HZRS3RCDXPVASNXV2QAA5NN3FJQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742074880 (29h34m4.62548851s) [noise=true , address=77.90.31.16:4734 , piece_id=EKFIFPNC3ZHIHPY7MKIDJPJ56WLK4GRTLXST3UP65VBNWCHYXXOA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742401104 (29h34m4.625814734s) [noise=true , address=45.91.116.12:5226 , piece_id=UHFYUUMMMT4O2KEKONRSZOJWWNU4XORJWG5MM4OAGROGRIOH7C7Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742765902 (29h34m4.626179531s) [noise=true , address=104.36.82.12:4150 , piece_id=BXYI6VIWSEUMGWTZ3PH4KLLOKF3Q3W4SQS2ZUHW2FHG2PLNWWFSA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.742900344 (29h34m4.626313975s) [noise=true , address=77.90.31.19:4842 , piece_id=GMN6WDKXDRQJIA2CQFE22ZJLN427DZ3W7KASUGXOGXNIBQDIST3Q, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743428240 (29h34m4.626841872s) [noise=true , address=45.91.116.7:4011 , piece_id=XQYPXJO3EC5XBADSTRKLI25T2ZB7TKVIHBU5INQK3VBADKSB54RQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743453449 (29h34m4.626867062s) [noise=true , address=45.91.116.9:5261 , piece_id=O2HIFND2QJPIJOSHIMRIFWKG7SJLOH2NED22RXL6ZSCII5NOP5DA, seg_pos={0 0}, upload=23763]: Done uploading piece. err: <nil>
Jun 24 14:45:04.743462590 (29h34m4.626876203s) [seg_pos={0 0}, upload=23763]: Segment reached optimal threshold of 13 pieces.
Jun 24 14:45:04.743646645 (29h34m4.62706026s) [noise=true , address=104.36.82.16:6293 , piece_id=N3O5FA4I62CVJHXNKR2SANMAFVRCIUBVUMK47R54QDMAK6E5FWLQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743789406 (29h34m4.62720304s) [noise=true , address=45.91.116.11:4118 , piece_id=THXTHC27TWOAGM4DW7MEFLP6X7C3MHGPLTPXVHAU56P35TKREMEQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jun 24 14:45:04.743867708 (29h34m4.627281324s) [noise=true , address=77.90.31.14:4663 , piece_id=CLMVCY7F6WPMJODOKHGUTFACMCPSKFH67WQFKEOCR75IIHQAR6XQ, seg_pos={0 0}, upload=23763]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: rpc: dial tcp 77.90.31.14:4663: operation was canceled; conn closed: op; piecestore: piecestore close: context canceled
Jun 24 14:45:04.743888405 (29h34m4.62730202s) [seg_pos={0 0}, upload=23763]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 24 14:45:04.743995945 (29h34m4.627409574s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 24 14:45:04.744001930 (29h34m4.627415543s) [upload=23763]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 24 14:45:04.894337720 (29h34m4.777751357s) [upload=23763]: Done uploading segments...
Jun 24 14:45:04.894379589 (29h34m4.777793218s) [upload=23763]: Upload finished. err: <nil>
Duration: 2.361346657s
core
uplink
us-select-1
|